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THE DEMONSTRABLE VALUE 
OF HONORS EDUCATION 


NEW RESEARCH EVIDENCE 


Introduction: 
The Demonstrable Value of Honors Education 


ANDREW J. COGNARD-BLACK 
St. MAry’s COLLEGE OF MARYLAND 


[I May of 2016, a small cadre of scholars was called to the campus 
of Wayne State University in Detroit, Michigan, for the Honors 
Education Research Colloquium, a two-day meeting focusing on the 
future direction of research in honors education. The participants 
were assembled by Jerry Herron, who at the time was president of the 
National Collegiate Honors Council (NCHC), close on the heels of 
a decision by the NCHC Board of Directors in June of the previous 
year to make research—along with professional development and 
advocacy—one of three strategic priorities. 

After a day of presentations, in turn, by each of the participants, 
the colloquium discussion turned on the second day to an enu- 
meration of ways in which the goal of encouraging honors research 
might best be effected. That enumeration included such topics as 
bridging the gap between those scholars doing related educational 
research inside and outside of honors and the establishment of an 
infrastructure to facilitate data collection and other collaborative 
research across multiple NCHC member institutions. One of the 
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concepts that emerged most forcefully from those discussions was 
vocal consensus about the need for more, and more robust, research 
evidence addressing the question of whether honors education adds 
value—for a society that helps to support the educational enterprise, 
for faculty and others who work to provide honors programming, 
for the institutions that house honors programs and colleges, and, 
especially, for the students who participate in such programs. 

Almost 100 years into the honors education experiment set in 
motion by Frank Aydelotte, there has been, of course, a great deal of 
research and writing about honors. In many ways, however, honors 
programs have for decades hummed along peacefully and without 
much notice from educational researchers or the watchful eyes of 
accreditors, and the notion that honors provides a better educational 
experience for high-ability and otherwise talented students has per- 
sisted largely as an unquestioned assumption. 

But that situation is changing. In the past few decades, as 
modernity unceasingly fetishizes anything presented numerically— 
especially if it comes in the form of a ranking—and as “assessment” 
has transformed from a buzzword into a bureaucratic juggernaut, 
more and more honors directors have been pulled into an inexora- 
ble vortex that ends each summer with an annual report filled with 
numbers and attending stories of honors student successes. Indeed, 
many honors administrators now routinely present their student 
accomplishments, rates of graduation, and other student successes 
compared to those in the general student body as evidence of honors 
program success. Many honors programs can show that their honors 
students graduate at higher rates than non-honors students, do so 
faster, graduate with higher GPAs, are more likely to go on to gradu- 
ate and professional school, and win prestigious national fellowships 
at higher rates. 

While the data points and success stories in annual reports 
may have their place on campus, from a research perspective these 
approaches often have serious limitations. The problem is that we 
want to know about the honors experience, but we often are mea- 
suring student characteristics, and doing so selectively. Most honors 
students, however, are starting at a different place than those in the 
general student body because the admissions processes for honors 
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at most institutions are designed to maximize the probability that 
unusually smart, talented, and motivated students enter those pro- 
grams. Thus, the evidence most often used to demonstrate the impact 
of honors programs is limited because it usually does not account for 
the differences that exist between honors and non-honors students 
at the moment of matriculation or point of entry into honors pro- 
grams. That reality makes it difficult to establish a causal connection 
between the honors experience and student change—we often do not 
have a really good handle on where the students started in order to 
evaluate how much they have grown. 

The problem at hand is one that has been of concern at least as 
far back as the very moment when NCHC emerged from the ashes 
of what had been, from 1957-1965, the Inter-University Commit- 
tee on the Superior Student (ICSS). The same year that NCHC was 
founded, Joseph W. Cohen (1966) published his monograph titled 
The Superior Student in American Higher Education chronicling the 
history and issues surrounding the efforts of the ICSS to expand 
honors education in the United States, and in the final pages Cohen 
turns his attention to honors program evaluation. While experi- 
ments employing random assignment of subjects to experimental 
and control groups are something of a gold-standard in explanatory 
research, Cohen notes that “no experimental attempts to determine 
objectively how attainment and achievements of honors participants 
compared to those of non-honors students are recorded” (Cohen 
1966:254). Indeed, while the design of such an experiment is not 
difficult to imagine, few would find acceptable any attempts at exper- 
imentation where some otherwise eligible students were denied 
entry into honors in the interests of demonstrating the unique effects 
of the program on student success. 

Yet there are other ways that researchers can capitalize on natu- 
rally occurring variation among honors and non-honors students to 
isolate the unique effects of honors program participation on student 
success and other meaningful outcomes such as civic mindedness, 
intellectual humility, or any variety of other outcomes that we might 
like for our students. Use of multivariate statistics and thoughtful 
research design that measures and statistically controls for relevant 
characteristics has become the bread and butter of research in the 
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social sciences, but such methods have been applied relatively rarely 
in research on honors education. 

In response to the outcry at the Detroit research colloquium, 
in early 2017 we announced a call for proposals for new research 
exploring the value added of honors programs. That announcement 
can be found in the Appendix to this volume, and the collection 
herein is the result of that call for new research on the demonstrable 
value of honors education. 

While many of the chapters rely on analytic methods that are 
more widely used in some fields than in others, authors have endeav- 
ored to include definitions and more explanation of statistical terms 
than one might typically find in a disciplinary journal where readers 
and writers share a common analytic frame and vocabulary. The col- 
lection begins with “Honors Value Added: Where We Came From, 
and What We Need to Know Next” by Hallie E. Savage, who provides 
an overview of the historical development of honors programs in the 
United States. In “History and Current Practices of Assessment to 
Demonstrate Value Added,” Patricia J. Smith then explores how the 
pursuit of evidence about value added in honors programs can be 
used within the program review process to inform change, and she 
points to the importance of collecting both quantitative and qualita- 
tive data in research investigating the benefits of honors education. 

In “Proving the Value of Honors Education: The Right Data and 
the Right Messaging,’ Bette L. Bottoms and Stacie L. McCloud then 
show how good data combined with simple yet compelling data 
summaries can be used on campus to illustrate the value that honors 
programs add for the larger institutions that house them, as well as 
for honors students in terms of classroom success and college com- 
pletion. Bottoms and McCloud explain how other honors program 
directors can engage in local collaborations to bring similar kinds 
of evidence to bear on their own campuses even when they may not 
have training in sophisticated statistical and quantitative research 
methods. In the process, they provide a template that readers from 
fields outside of the social sciences may find more accessible, and 
in so doing they provide a comfortable launchpad that propels the 
volume forward to subsequent chapters. 
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The remainder of the collection features a variety of formal 
research contributions that make use of rigorous multivariate and 
other research methods designed to isolate the unique effects of hon- 
ors program participation on student success, and thus to bolster the 
accumulation of evidence on the question of the value added from 
honors education. Dulce Diaz, Susan P. Farruggia, Meredith E. Well- 
man, and Bette L. Bottoms anchor the assemblage in a chapter that 
boldly claims, “Honors Education Has a Positive Effect on College 
Student Success.” Using data on over 20,000 students collected during 
the period 2006-2012 at a large public university, they found signifi- 
cant benefits to student success from participating in the university 
honors college. After controlling for various pre-matriculation vari- 
ables, participation in the honors program was positively associated 
with first-term GPA, first-year credits earned, second-year retention, 
and graduation rates. Notably, they also found that those associa- 
tions were stronger for underrepresented minority students on some 
success indicators, suggesting that honors education may help to 
address race and ethnicity achievement gaps that we witness else- 
where in higher education. 

Katie Patton, David Coleman, and Lisa W. Kay’s “High-Impact 
Honors Practices” details how they utilized Astin’s “inputs-envi- 
ronment-outputs” (I-E-O) model to examine how the environment 
of the Eastern Kentucky University Honors Program affects stu- 
dent outcomes. Using data from almost 600 honors students and a 
comparable group of non-honors students with similar academic 
preparation, they found that honors students had higher retention 
and graduation rates than the comparable non-honors students. 
Moreover, they found that there were higher retention and gradu- 
ation rates among those honors students who were more highly 
involved in the high-impact practices that are an important feature 
of the honors program. 

In “GPA as a Product, Not a Measure, of Success in Honors,” 
Lorelle A. Meadows, Maura Hollister, Mary Raber, and Laura Kas- 
son Fiss describe the unique features of the Pavlis Honors College 
at Michigan Technical University, where any student is free to join 
honors regardless of GPA. Unlike much of the other research that 
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focuses on outcomes such as retention and graduation rates, Mead- 
ows et al. used data from initial and final written reflections by 26 
students in their first honors seminar to evaluate the development of 
“self-authorship,’ fostering students’ authentic internal voices, as the 
outcome of interest. They found that the unique approach of their 
courses was associated with higher levels of self-authorship, and they 
report: “Self-authorship development has been shown to produce 
graduates who are better prepared to manage adversity and change, 
make meaningful decisions, benefit from their educational experi- 
ences, and learn deeply throughout their adult lives” (p. 143). 

Art L. Spisak, Robert F. Kirby, and Emily M. Johnson present 
evidence of value added from a slightly different vantage in “Adding 
Value through Honors at the University of Iowa.” Using data from 
over 3,000 students at the University of Iowa, they compared hon- 
ors students who lived in honors housing and/or participated in an 
honors pre-semester credit-bearing class with similar honors peers 
who did not opt for those experiences. Results indicate that students 
who elected to participate in a pre-semester honors class and live 
in honors housing were more engaged in the honors program and, 
moreover, had greater academic success as measured by outcomes 
such as cumulative GPA at the end of the first year and completion 
of honors requirements. The comparison—not of honors with non- 
honors but of students with varying levels of engagement within 
the honors program—suggests that it may be exposure to specific 
features of an honors program that adds value to the educational 
experience. 

Robert D. Brown, Jonathan Winburn, and Douglass Sullivan- 
Gonzalez then discuss evidence of value added both for individual 
honors students and for the institution in “The Value Added of Hon- 
ors Programs in Recruitment, Retention, and Student Success.” They 
used survey data from over 500 honors students to evaluate the ways 
in which the honors college adds value at the institutional level to 
the University of Mississippi. They found that the enhanced aca- 
demic environment resulted in a significant recruitment impact for 
the university that also helps to mitigate against brain drain whereby 
the best students leave the state to pursue schooling elsewhere. In a 
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supplemental analysis, they used a matched pairs approach for over 
1,500 honors students and comparable non-honors students and 
found further evidence that honors programs contribute to higher 
student GPAs and greater retention in each subsequent year of the 
students’ college careers. 

Jane B. Honeycutt contributes an important aspect to the collec- 
tion by exploring the value added of honors at two-year institutions 
in “Community College Honors Benefits.” She compared outcomes 
for 95 honors program participants at Northeast State Community 
College in Tennessee with those for 357 academically matched peers 
who did not participate in honors. She used a propensity score match- 
ing process to control for confounding variables such as high school 
GPA, ACT score, parental income, and several other important 
background characteristics. Honeycutt found that honors program 
participants, compared to non-honors students, earned significantly 
higher grades in their English Composition II course, earned signifi- 
cantly higher GPAs, and were significantly more likely to graduate. 

The final research article in this collection comes from George 
Smeaton and Margaret Walsh at Keene State College. Their essay, 
“Contributions of Small Honors Programs,” presents data from 
approximately 100 honors students and a comparable group of stu- 
dents who received merit-based scholarships but did not participate 
in honors. Like other authors in this collection, they found that hon- 
ors students had higher retention rates and greater involvement in 
high-impact educational practices, but they also present qualitative 
data that suggest that specific program features, such as an honors 
living-learning community and an emphasis on experiential learn- 
ing, contributed to those improved outcomes. 

In the final essay, co-editor Jerry Herron and his collaborator 
D. Carl Freeman ask, “What Next?” They provide a synthesis of the 
cumulative contributions of the collection as well as offering sugges- 
tions for future research directions. 

We believe that together the contributions in this collection 
provide important answers and compelling evidence that honors 
programming does contribute something above and beyond what 
honors students themselves bring to the educational experience. This 
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research also presents a useful cross section of research methods that 
we hope will inspire future research efforts in this area of inquiry. 
Moreover, we hope that the cover image from Vincent van Gogh's 
Olive Orchard will provide a visual reminder of the power and beauty 
that are possible when conscientious stewards provide the right con- 
ditions for growth. 

The results presented in this volume are a forceful answer to the 
question of whether honors adds value, and the evidence indicates 
that the answer to the question is yes. Using a variety of different 
methods and exploring a variety of different outcomes across a 
diversity of institutions and institution types, honors programming 
adds demonstrable value for the students who participate. While it 
is true that those students tend to start college in a stronger position 
in terms of academic preparation—as indicated by factors such as 
college entrance tests and high school GPA—meaningful evidence 
demonstrates that honors programs propel those students further 
than they would have gone without those programs. Yet research evi- 
dence is rarely unambiguous, and results are often qualified by the 
parameters of sampling designs and other methodological choices. 
Readers must discern for themselves just how demonstrable the 
evidence is and what directions future research should take, and so 
I encourage readers to investigate for themselves in the pages that 
follow. 
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Honors Value Added: 
Where We Came From, and 
What We Need to Know Next 


HALLIE E. SAVAGE 
CLARION UNIVERSITY OF PENNSYLVANIA 


THE VALUE ADDED IMPERATIVE 


he pressure is on, and growing greater when it comes to 

defining, disseminating, and defending the value of higher 
education generally and the reasons for funding it (Harnisch 2011). 
Complaints abound regarding the rising costs of higher education, 
and many legislators and the public are demanding accountability. 
Funding cuts are forcing many colleges and universities to priori- 
tize and to evaluate what merits support and what does not. As a 
part of a large array of undergraduate programs, honors programs 
and honors colleges face increasingly greater pressure to justify 
their existence. 

That said, honors programs and colleges are in a good position 
to make a case for the value that honors adds to institutional out- 
comes. Honors education is known nationally and internationally 
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for leadership in high-quality undergraduate programs. Honors 
faculty enjoy the opportunity to create unique and innovative 
learning environments, with academically talented undergraduate 
students as the immediate beneficiaries. Institutions benefit from 
recruitment of ambitious, motivated students who typically have 
higher retention and graduation rates when compared to those in 
the traditional student population. Yet despite these obvious insti- 
tutional benefits, questions persist regarding the value that honors 
adds and how precisely that value is to be measured. 

The term “value added” has emerged in higher education in 
reference to models that can be used to evaluate, monitor, and 
improve an institution (Kim and Lalancette 2013). Institutional 
outcomes have mainly focused on student performance measures 
such as scores on standardized tests or the percentage of students 
progressing to higher levels of education. Student performance 
can also be measured through group metrics such as retention 
and graduation rates. Our outcomes-based culture is driven by the 
need for assessments of value added that capture demonstrably the 
impact institutions have on improving student performance. An 
institution’s achievement on performance indicators may be signifi- 
cant for funding purposes—e.g., state appropriations—so that such 
measures become crucial to a school’s fiscal health. 

The value of honors programs and colleges and, consequently, 
the contribution of honors to the institution are enthusiastically 
articulated by honors deans, directors, and college presidents. In 
2015 the Journal of the National Collegiate Honors Council included 
a forum titled “The Value of Honors,’ in which a cadre of higher 
education leaders described the benefits of honors programming 
and the contribution of such programming to institutions of higher 
education. As a president with a long tenure in higher education, E. 
Gordon Gee provided this view: 


I have been around the block for the last thirty-plus years 
serving as president of five major institutions in the United 
States, and I can affirm that the increased value placed on 
an honors education is enriching entire universities and 
how they operate. (Gee 2015:177) 


14 


NEED TO KNOW NEXT 


A host of other leaders affirmed Gee's point. The contributions 
of honors to the university-wide curriculum is characterized as a 
significant benefit, as Jake B. Schrum, President of Emory & Henry 
College, and Joe Lane, Director of the Honors Program, affirmed: 


Our honors program has made it possible for us to raise the 
level of discussion in classes across the curriculum and has 
revealed opportunities for investing all of our students in 
projects that will widen their horizons and allow them to con- 
tribute to positive social change. (Schrum and Lane 2015:39) 


These statements testify to the value honors contributes to under- 
graduate institutions. High-quality programs are those defined by 
the creation of communities of students, faculty, and administrators 
investing considerable time and effort building learning communi- 
ties. Honors education insists on the construction and sustenance 
of the highest quality participation in teaching and learning. A use- 
ful perspective on the value honors adds to the institution can be 
gained from a review of the growth of honors programs and colleges 
within undergraduate institutions. 


A BRIEF HISTORY OF THE HONORS TRAJECTORY 
WITHIN INSTITUTIONS 


For over 200 years, honors programs and colleges have expe- 
rienced phenomenal growth within undergraduate education. The 
earliest reports of the approach that has come to be defined as honors 
were of the “pass-honors” reconstruction of the grading structure at 
Oxford University in 1804. This grading process was intended to 
expand university-wide curricular focus and support independent 
student research (Standley 1993). In the early 1920s, the formal 
programmatic establishment of honors in the American university 
infrastructure occurred when Swarthmore College replaced tradi- 
tional coursework with honors courses. The first formalized honors 
courses replaced upper-division course offerings and revealed early 
emphasis on writing, critical thinking, and capstone experiences 
(Guzy 2003). 
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In 1924, Joseph W. Cohen, a philosopher, arrived at the Univer- 
sity of Colorado to begin his tenure. Cohen reported later that he 
was angered “by the inertia I witnessed before the deep problems of 
quality in a state institution confronted by numbers, by routine, by 
the recalcitrance of legislatures” (Cohen 1966:viii). Cohen observed 
that students with high grades lacked knowledge and preparation, 
and he believed that university faculty should be empowered to 
produce intellectuals. Further, he was convinced that public edu- 
cation could equal the best offered at elite institutions, that these 
schools did not have a monopoly of faculty or student intellect. 
Based on these philosophical underpinnings, Cohen established an 
honors program that served as a model for others that followed. 

Formal honors program development in American higher 
education accelerated following World War II. Programs were 
established with the intent of raising the academic standards for 
undergraduate education. A desire for rigorous standards for all 
education resulted from national competition for global leadership, 
i.e., Sputniks 1 and 2, as well as large increases in new student enroll- 
ment (Andrews 2011). Andrews provides an enriching description 
of how these societal forces influenced the formalization of hon- 
ors and improving educational quality. The oldest view of program 
quality is one that suggests that coherent and rigorous curriculum 
requirements are at the core of high-quality programs. Administra- 
tors and faculty in honors leadership united in their dedication to 
high-quality programs and formed the National Collegiate Honors 
Council (NCHC). 

The first NCHC conference was held at the University of Kan- 
sas on October 22-24, 1966, with a program structured around five 
invited presenters and a student panel. Of the five papers, two pre- 
sentations were on the motivation of honors students in colloquia 
and courses. Walter Weir (1966), then director of the honors pro- 
gram at the University of Colorado, described honors students in 
this way: 


our honors students come to us highly motivated to suc- 
ceed, to climb the ladder of affluence and success. They 
tend to have more intellectual curiosity than most students, 
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to be quicker and more industrious; but their most funda- 
mental trait is their ability to get good grades. (p. 45) 


This description was supported by Dean James Olsen (1966) from 
Kent State University, who focused on the “intrinsic” and “extrinsic” 
needs of honors students. His research was intended to assess the 
worth and effectiveness of honors courses as well as other features 
of the Kent State Honors College. Over a span of six years, honors 
students were asked to describe an ideal honors course in compari- 
son to existing courses within their honors college. The assumption 
was made that students with attributes defined by admissions crite- 
ria (intrinsic qualities inherent in the student performance) would 
achieve success in a prescribed learning environment (as defined by 
extrinsic qualities within the learning context). Olsen (1966) con- 
cluded the following from his research: 


if honors students are to be motivated and to be satisfied in 
honors courses, very careful attention must be given to the 
instructors and the methods of instruction employed ... [;] 
what is necessary is to recognize that the students are not 
completely self-generative and that the primary consider- 
ation is the instructor and his methods. (p. 56) 


Considered collectively, these early studies suggest that honors 
programs and colleges may predict outcomes (e.g., academic per- 
formance, program completion rate) based on intrinsic criteria or 
performance measures; however, an essential characteristic not 
easily measured is motivation and persistence in problem solving. 
Honors learning environments were, and continue to be, developed 
to facilitate, motivate, and nurture a drive for academic challenge. 
In 1967, the National Collegiate Honors Council Annual Con- 
ference was jointly hosted by Catholic University, Georgetown 
University, Howard University, the University of Maryland, and the 
U.S. Office of Education in Washington, DC. With the proliferation 
in honors programs and colleges, the honors conference agenda 
increased its focus on honors/non-honors comparisons. One group 
of research presentations at the 1967 meeting categorized gifted 
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superior performance as a “domain” in areas of science, technology, 
engineering, and mathematics. Vernon Williams (1967) presented 
a pilot study at the University of Nebraska that compared honors 
seniors with high-ability non-honors seniors in agriculture. This 
pilot study dealt with student-faculty engagement, appreciation 
of the scientific and professional nature of agriculture, student 
involvement in academic endeavors, and rational thinking about 
occupational development. Williams concluded that the honors 
students in agriculture had more interaction with faculty, felt more 
positively about their interaction, and were more involved with 
academic work when compared with the high-ability students who 
did not participate in the honors program. These early honors edu- 
cators acknowledged that students and faculty work collaboratively 
to create the honors learning community. 

Over the next several decades, honors education emphasized 
innovative curricular development undergirded by critical thinking 
and active engagement in community-based learning approaches 
such as City-as-Text™ (Long 2015). The institutional benefit broad- 
ened, and honors came to be viewed as a laboratory that influenced 
the traditional curricula and elevated the rigor of the undergradu- 
ate experience across campus. Furthermore, student performance 
outcomes such as grade point averages, test performance, and a pro- 
clivity for academic challenge positively impacted the institutional 
profiles and the academic atmosphere of the institution generally. 
The proliferation of honors programs and colleges was followed by 
a slower rate of growth (Smith and Scott 2016). This trend in the 
1970s and 1980s is most likely correlated with the end of expansion 
in higher education and severe budget restrictions. 

Schools continued to develop honors programs, and some 
were transformed into honors colleges. Program quality—how to 
enhance it and how to evaluate it—became an important priority 
for honors educators. As forces external to higher education pressed 
for accountability, many institutions began to look more critically 
at their programs to decide which should receive continued fund- 
ing. In 2005, the editor of the Journal of the National Collegiate 
Honors Council accepted the challenge to elucidate the definition 
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of honors, and the journal invited manuscripts for a special forum 
dedicated to this pivotal question: “What is Honors?”: 


While it is hard to find any single characteristic that dis- 
tinguishes honors from non-honors students, teachers, or 
courses, and while honors programs/colleges across the 
country are far more different from each other than are, 
for instance, English departments or service learning pro- 
grams, we do share one trait with passion and, I daresay, 
universal agreement: our belief in the vitality and necessity 
of outstanding undergraduate education. (Long 2005:9) 


In other words, honors educators are characterized by a drive to 
challenge students beyond the traditional requirements, regardless 
of the discipline. Honors educators continued to evaluate com- 
monalities and what was frequently termed “academic excellence.” 
In pursuit of this goal, the NCHC Board of Directors eventually 
approved an official definition of honors: 


Honors education is characterized by in-class and extra- 
curricular activities that are measurably broader, deeper, or 
more complex than comparable learning experiences typi- 
cally found at institutions of higher education. (NCHC 2013) 


Historically, honors communities are the product of faculty 
and administrators actively designing and implementing tangible 
program requirements that substantially enhance the quality of 
student learning. Considered from a university-wide perspective, 
honors plays an institutional leadership role for curricular devel- 
opment and pedagogical approaches that influence high-quality 
learning outcomes. 


THE INHERENT VALUE OF STUDENT ENGAGEMENT 


Active student engagement is a hallmark of high-quality pro- 
grams (Pascarella and Terenzini 2005). Research has documented 
that student learning and development are enhanced when students 
become actively involved in out-of-class activities with peers and 
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faculty mentors (Kuh 1993; Kuh, Schuh, and Whitt 1991). From its 
inception, active student engagement in forums and presentations 
was recognized as a core value of NCHC conferences. Diverse and 
engaged students remain vital to the development and sustenance 
of honors programs and their engagement across campus. 

Given the role of honors education in nurturing student suc- 
cess and its positive contribution to institutional performance 
measures, further research is needed to explore student engage- 
ment both quantitatively and qualitatively. Generally, research has 
sought to determine intrinsic variables (i.e., student characteris- 
tics) and their role in predicting academic success. These research 
designs are complicated by student diversity across different types 
of institutions and involvement in a wide variety of program- 
matic requirements. It is equally complicated to design studies that 
describe the inherent passion or propensity for academic challenge. 

Conceptualizations of honors students emphasize the role of 
drive or persistence for academic challenge. That is, students of sim- 
ilar abilities can be characterized by their persistence in performing 
a task and/or the drive to achieve it. Literature in elementary and 
secondary gifted education, for example, underscores the impor- 
tant role of motivation in academic performance. Renzulli (1986) 
investigated task commitment as a central component of giftedness 
along with above-average ability and creativity. Terman and Oden 
(1959) noted that the most successful of their subjects could be dis- 
tinguished from less successful subjects of equal ability by their task 
persistence. Therefore, measures of motivation and task persistence 
may be revealing for undergraduate honors education. That is, grad- 
uates of honors programs are often distinguished by their acceptance 
of challenge and drive to achieve beyond the minimal academic 
requirements. Clinkenbeard (1996), for example, cited studies that 
compared subgroups of the gifted on motivation type or style. Stu- 
dents with a high proclivity for academic success when compared 
to peers with average motivation have been studied longitudinally. 
Students who are highly motivated may perform differently, in dis- 
tinctive ways, from those who lack motivation but who are similarly 
identified as innately intelligent. These investigations, then, would 
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seem to be applicable to undergraduate honors education; that is, 
results would describe motivational processes that affect student 
performance outcomes. Qualitatively, honors students infuse aca- 
demic excellence into university-wide classrooms by demonstrating 
a passion for challenge, curiosity, and diligence. 


HONORS AS ADDED VALUE TO INSTITUTIONAL OUTCOMES 


Investigations have been designed to describe the value-added 
impact of honors by comparing honors with non-honors student 
outcomes. Cosgrove (2004) studied the academic performance, 
retention, and degree-completion rates of three groups of students: 
those who completed the honors program, students who partici- 
pated in the honors program but did not complete the program, 
and students who qualified for honors but were not enrolled in 
honors. The student data were gathered across institutions in the 
Pennsylvania State System of Higher Education. The study con- 
trolled for the effects of student, institutional, and honors program 
characteristics. To examine the retention and degree completion 
rates, the research was gathered longitudinally over a five-year 
period. Results of this study revealed that honors program com- 
pleters had the highest academic performance and graduation rates 
and the shortest time to degree compared with non-honors peers, 
including those partially exposed to honors. Support from other 
inter-institutional investigations is needed to describe the value of 
honors for diverse types of institutions and for a range of qualitative 
variables such as motivation and variation in characteristics such as 
gender, race, and other forms of diversity. 

Studies were also designed to explore prediction of honors pro- 
gram completion. Evidence is building that high school grade point 
average (GPA) is the most significant predictor of honors program 
completion (McKay 2009; Savage, Raehsler, and Fiedor 2014). 
Grade point average appears, then, to be an important factor for 
admissions standards. As an institutional performance indicator, 
program completion rate makes a significant contribution to quan- 
titative performance indicators tied to institutional success such as 
retention and graduation rates. 
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Other research has examined the impact of honors over the 
course of program participation. Hébert and McBee (2007) stud- 
ied the qualitative enrichment of honors education over an entire 
undergraduate program of study. The results identified key social 
and intellectual bonding through the honors program that sup- 
ported the unique needs of students. Other research was designed 
to answer questions such as whether students who participate in 
an honors program have higher retention and graduation rates 
in comparison to similar nonparticipants. Results revealed that 
participation in honors related to retention differences over time. 
Although data collection was limited for examining four-year 
graduation rates, Slavin, Coladarci, and Pratt (2008) hypothesized 
that if retention rates continued to increase over time, they would 
ultimately positively influence graduation rates. Collectively, these 
group performance studies provide quantitative support for how 
honors programs and colleges positively influence institutional 
performance outcomes. A scarcity of data, however, exists on the 
qualitative impact honors provides for the institution, evident in the 
presidential testimonies mentioned earlier. Such quality measures 
might include how honors transforms both the learning environ- 
ments when it serves as a curricular laboratory for the campus and 
the quality of student engagement in classrooms across the curricu- 
lum through the honors students’ passion for academic challenge. 

Academically talented high school students are matriculating to 
universities based on their desire for academic quality at competi- 
tive prices. The institution's challenge is to engage faculty with these 
students in rigorous learning experiences that prepare students for 
professional and personal success and responsible citizenship in 
cost-effective ways. Honors education, viewed from an outcomes- 
based perspective, recognizes the need for measures of value added 
that capture the causal influence of institutions on their students. 
The very existence of honors programs and colleges is dependent on 
research that documents value in the face of internal competition 
for institutional resources. What measures are selected, when the 
measures are best applied, and detailed demographic descriptions 
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are probably best considered in the context of a national discussion 
among researchers, honors professionals, and organizations such as 
NCHC. Discussion might also include more widespread use of an 
honors designation in institutional student databases. An honors 
designation, if universally applied in institutional data management 
systems, can readily capture outcome measures such as program 
completion rates and graduation rates. An honors designation can 
also reveal outcomes observed only once, such as persistence or 
graduation. Future research is needed to address these issues in a 
more systematic way to quantify and qualify the value of honors 
programs and colleges for higher education institutions. 

The research found in this monograph moves us forward 
toward achieving our research goals. The future of honors educa- 
tion is dependent on more robust research such as that emerging 
from the NCHC research colloquium that took place at Wayne 
State University. Such projects elucidate the value of honors educa- 
tion and are essential if honors programs and colleges are to survive 
and mature. In turn, universities and colleges benefit, quantitatively 
and qualitatively, from investments in honors communities. 
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Wwe more than 1,500 honors programs currently in operation 
and hundreds of millions of dollars being spent throughout 
American institutions, external pressure is building for account- 
ability in honors programs (Scott and Smith 2016). Today’s society 
“expects colleges and universities to graduate students who can get 
things done in the world and are prepared for effective and engaged 
citizenship” (Keeling et al. 2004:5). Doyle (2004) also has noted the 
increasing scrutiny of higher education: 


the attention given to higher education's success at fostering 
student learning has increased in recent years. The rapidly 
rising cost of higher education and the increased attention 
to accountability only add to the pressure on colleges and 
universities to validate their lofty claims of higher learning. 
(p. 375) 


Scott Carnicom and Christopher A. Snyder (2010) suggest that 
honors programs and colleges that do not participate in assessment 
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risk alienating the “accountability-driven entities” within the insti- 
tution and the larger higher education community (p. 69). Higher 
education institutions continue to focus on data such as college 
grade point averages (GPA) and retention and graduation rates as 
determinants of success. Honors programs are admitting students, 
however, who are already expected to do well in higher education 
based on their prior high school success as measured by GPAs 
and standardized test scores. The use of these same types of data, 
therefore, is insufficient to evaluate the value added by an honors 
program or college. In 2006, Achterberg argued for the importance 
of research and accountability for the survival of honors, and very 
little has changed in over a decade. Achterberg (2006) stated: 


At present, there is little understanding of honors issues 
in higher education and few studies that show its worth 
one way or the other. Honors cannot survive the future on 
anecdotal evidence. If we do not act, and if we do not lead, 
there will be nothing in honors to save at all. (p. 39) 


Sean K. Kelly (2013) suggests that rates of honors program 
completion should not be the only measure of success for an 
honors program and that sometimes, in fact, the high standards 
of the honors program could work against it, but he also points 
out that that does not mean the program has not been success- 
ful. Kelly (2013) argues that honors administrators should instead 
be assessing how students’ involvement in honors correlates with 
engagement in university life, accomplishments after graduation, 
and higher satisfaction with their university experience, in addition 
to higher graduation rates. He agrees that these assessments could 
prove useful in promoting the position of honors within the institu- 
tion. Because college and university administrators expect honors 
programs and colleges to provide evidence of added value to the 
students’ academic careers through participation in such programs, 
greater involvement in outcomes assessment could be the answer 
that honors programs are seeking. 

Assessment and evaluation in honors programs can serve 
multiple functions. By undergoing the process of assessment and 
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evaluation, programs can respond to concerns about and demands 
for accountability from internal and external audiences (Achterberg 
2006). Internal audiences include college or university administra- 
tion; external audiences include students, families, alumni, donors, 
and other taxpayers. More importantly, the use of assessment can 
contribute to a “shared understanding” of the values and mission of 
a program and lead to more informed decision-making and trans- 
parency (Jones and Whelburg 2014:18). 

As the first wave of honors programs was being introduced at 
institutions of higher education in the first half of the 20th century, 
evaluation of these programs was “for the most part . . . subjec- 
tive and nonscientific” (Heist and Langland 1966:257). According 
to Heist and Langland (1966), early evaluations “rarely extended 
beyond assignment of grades for performance” (p. 254). Evaluation 
efforts tended to be qualitative assessments that focused on student 
and faculty opinion rather than student outcomes and program 
quality (Heist and Langland 1966). When the Inter-University 
Committee on the Superior Student (ICSS) was established in 1956, 
attention was quickly given to the evaluation of existing programs, 
and by 1961 the ICSS invited the Social Science Research Council's 
Committee on Personality Development in Youth to lead a research 
conference (Heist and Langland 1966). This event marked a transi- 
tion in evaluation for the field of honors education, and the studies 
that began appearing in The Superior Student, ICSS’s publication, in 
the following decade, although still largely focused on student and 
faculty opinions, began to generate quantitative data that allowed 
programs to examine selection and retention issues (Heist and 
Langland 1966). 

By 1963, 56 percent of honors programs reported using “simple 
statistical examinations of the number of dropouts from their pro- 
grams...” and “almost half of all directors reported that their staffs 
had conducted or begun formal evaluations” within the five years 
prior (Heist and Langland 1966:265). Despite the progress, Heist 
and Langland (1966) stated that “the evaluative research conducted 
on honors programs to date [had] not provided a very adequate 
basis for ascertaining the real values, as well as the weaknesses, of 
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special programs for the superior or gifted student” (p. 274). Heist 
and Langland (1966) argued that additional evaluation needed to 
occur and that the particular variables that needed further attention 
included program participants, faculty, course content, physical 
and social context of the program, and the cost involved in attain- 
ing objectives. 

Two early works about honors assessment and evaluation were 
by C. Grey Austin. He published Handbook for the Evaluation of an 
Honors Program in 1981, and in 1991 Austin’s Honors Programs: 
Development, Review, and Revitalization provided extensive details 
about the self-study process and the process for undertaking a pro- 
gram review. The first faculty institute on honors education was 
hosted by NCHC’s Committee on Honors Evaluation in 1997 and 
had 76 participants (Brown 1997). At this institute, the committee 
introduced the ideas of external review and self-study. John Grady, 
co-chair for the Committee on Honors Evaluation, said that these 
two approaches were the most effective ways to improve an honors 
program, and he was quoted as saying, “assessment is the means; 
improvement is the end” (Grady cited in Brown 1997:1). 

The focus on periodic evaluation shifted to systematic, ongoing 
assessment, and the first handbook was replaced by Otero and Spur- 
rier’s (2005) Assessing and Evaluating Honors Programs and Honors 
Colleges: A Practical Handbook, also published by NCHC. Figure 
1 presents a diagram illustrating their approach to assessment and 
evaluation. In this monograph, Otero and Spurrier explain that 
in order to conduct effective assessment, as can be seen in Figure 
1, the honors director or dean must first identify outcomes spe- 
cifically related to the educational objectives of the program. The 
director must then gather and interpret evidence of how the pro- 
gram is or is not meeting those outcomes. Finally, after interpreting 
the evidence to determine whether or not the program is meeting 
its defined objectives, the director and faculty should implement 
any change that is needed. Otero and Spurrier (2005) provide hon- 
ors administrators with information on how to evaluate and assess 
their programs using program self-study and a site visit in order 
to “demonstrate their strengths, address their weaknesses, generate 
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institutional support, and gain outside validation of their accom- 
plishments and goals” (p. 5). 

Otero and Spurrier (2005) suggested that an important compo- 
nent for assessing an honors program or college is the assessment of 
student learning. Assessment of student learning “is most effective 
when it reflects an understanding of learning as multidimensional 
and revealed in performance through time” (Otero and Spurrier 
2005:9). This type of assessment also requires the use of both quan- 
titative measures, such as student grades or credit hours earned, 
and qualitative measures, such as the review of a portfolio or cap- 
stone project. 

In 2011, the Assessment and Evaluation Committee of NCHC 
issued a report, written by Otero, Spurrier, and Lanier, that focused 
on quantitative, measurable data. Their reason for such a focus was 
that they believed that “evaluation provides an opportunity for hon- 
ors programs and honors colleges to demonstrate their strengths, 
address their weaknesses, generate institutional support, and gain 
outside validation of their accomplishments and goals” (p. 12). The 
committee report also suggests that the assessment process, which 


Figure 1. MetHops oF ASSESSMENT EFFECTIVENESS 


Honors Mission 
Educational Objectives 


Identify Outcomes Gather Evidence Interpret Evidence 


Implement Change 


Note: Reproduced from Otero and Spurrier (2005). 
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is presented in Figure 2, is not just a collection of methods: it is 
cyclical in nature (Otero, Spurrier, and Lanier 2011). 

Although assessment and evaluation have been widely adopted 
for the continuous improvement of academic programs within the 
larger realm of higher education, not all faculty and administrators 
in honors education are accepting of these practices. Joan Digby 
(2014) argued against outcomes assessment in higher education, 
stating that her “goal is not to score or measure students against 
preconceived expectations but to encourage the unexpected, break- 
through response that is utterly new, different, and thus exciting” (p. 
4). The idea that student learning is not easily measured is an idea 
that remains prevalent within the field of honors education despite 
a long history of outcomes assessment being practiced within the 
field. Carnicom and Snyder (2010) suggest that the tasks involved 


Figure 2. FEEDBACK CYCLE OF AN ASSESSMENT PLAN 


Identify 
Use Assessment Results to Revise Student Learning Goals for 
Goals, Teaching Methodologies, Courses/Majors 
Curriculum, and Budgeting Priorities 


Design Course/Major 
Curriculum to Meet Goals 


Assess Student Learning to 
Identify Needed Revisions 
to Course/Major 


Note: Reproduced from Otero, Spurrier, and Lanier (2011). 
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in assessing student learning are a distraction for faculty from their 
primary responsibilities of teaching and research within their own 
discipline. 

In response to these types of criticism, Driscoll (2011) con- 
ducted research based on her interest in learning about the extent 
to which the evaluation of honors programs was actively underway 
throughout the nation and whether programs had remained hesi- 
tant to adopt such practices. Driscoll (2011) conducted a survey 
of honors program administrators and learned that of the 38 par- 
ticipants, 57 percent did conduct at least some assessment of their 
program. Of the 43 percent who did not currently participate in 
assessment, 40 percent reported having a plan in place to intro- 
duce assessment. Other reasons given for not conducting program 
assessment included having a new program, a new administrator, 
or insufficient time. Although less than a quarter of participants 
voiced a philosophical opposition, Driscoll (2011) concluded that a 
consensus had not yet been reached among the honors community 
on the value of assessment as a quality assurance practice. Driscoll 
(2011) argued that the greatest need in the honors community in 
regards to assessment is “research to determine effective and reli- 
able program measures” (p. 101) and how to apply them. 

Very little discussion has taken place on the national level 
regarding the assessment practices currently being used through- 
out honors education nationally, so it is not widely known whether 
honors programs are engaging in program assessment or whether 
the results of program evaluations are being used to inform changes 
within the program. To better understand the extent to which 
honors programs are actively involved in program assessment, I 
designed a study to examine how actively program assessment is 
being conducted and what types of practices are being employed to 
examine student learning specifically. Having this knowledge will 
inform the national honors community about what work we still 
have ahead of us before we can create a culture of assessment within 
honors education. 
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METHODOLOGY 


This study examines the extent to which and through what 
means individuals actively involved in the leadership of honors 
education are utilizing assessment of student learning. Specifically, 
a survey consisting of 26 items was distributed in fall 2016, using 
Qualtrics, a survey software designed to administer surveys elec- 
tronically and to securely store the results. The survey collected 
data from respondents regarding their demographic information 
(age range, gender, race-ethnicity, and educational level). Respon- 
dents were asked to report their position within honors as well 
as their institutional type, honors program type, honors program 
size, and their years of experience in honors education. Respon- 
dents were then also asked to record the types of assessment, such 
as individual assignments, theses, capstone projects, and portfolios, 
utilized in their programs. 


Overview of Study Participants and 
Recruitment Procedures 


Selection of participants for the study was done using the pur- 
posive sampling approach. Participants were recruited based on 
their experience and active leadership in honors education. Specifi- 
cally, the survey was distributed to 838 participants, each of whom 
is listed as serving as a current head of an honors program or col- 
lege that is affiliated as an institutional member of NCHC. 

Of the 838 individuals invited to participate, a total of 273 com- 
pleted the survey for a total response rate of 32.6 percent. Of the 
273 participants, 220 (80.6%) held a doctoral degree. A little more 
than half of the participants had five years or less experience as an 
honors administrator (54.5%) while 17.2 percent had ten years or 
more. Seventy-four (27.3%) were working at honors colleges, 197 
(72.2%) were at honors programs, and two (.5% identified them- 
selves as being with other. 
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RESULTS AND DISCUSSION 


Program deans and directors reported a variety of types of stu- 
dent work that is being collected for use in outcomes assessment. 
Of the 245 respondents to this question, 126 (51.4%) collect indi- 
vidual student assignments, 116 (47.3%) assess a student thesis, 
106 (43.3%) use a capstone project, 100 (40.8%) report using writ- 
ten compositions or research papers, 76 (31%) assess participation 
in internship or field experience, 69 (28.2%) collect portfolios, 39 
(15.9%) utilize questions embedded in larger assignments, and 
36 (14.6%) use exams. Other methods of assessment that were 
reported include independent project proposals, student survey 
instruments, exit surveys, alumni surveys, graduation and reten- 
tion rates, student grades, student conference presentations, and 
student self-assessment of skill development. 

Overall, it looks as if there has been an increase in the number 
of honors programs that are participating in outcomes assessment 
since Driscoll’s 2011 study. Of those responding to the survey in 
this study, 63 percent say that their honors program has defined 
learning outcomes as part of a programmatic outcomes assessment, 
and 77 percent say that their program participates in discussions 
of programmatic outcomes assessment at least once per year. 
Additionally, 61 percent of participants reported that outcomes 
assessment findings are used in the analysis of program policies 
and procedures. 

The outcomes assessment process as described by Otero and 
Spurrier (2005) involves multiple steps including identifying out- 
comes, gathering evidence, interpreting that evidence, and then 
using that evidence to implement change. To analyze the efficacy or 
level of preparedness that participants feel for participating in each 
step of the process, respondents were asked about their knowledge 
and proficiency. Regarding the first step of identifying program 
outcomes, 73 percent of participants feel that they are knowledge- 
able in the process of creating program-related outcomes. The 
next phase in the process is gathering evidence related to those 
outcomes. Eighty-four percent of respondents reported that they 
were capable of gathering appropriate evidence to be used to assess 
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program-related outcomes. Once that evidence is gathered, the 
honors director or dean must know how to interpret that evidence 
before it is useful to the program, and 73 percent believe that they are 
proficient in interpreting evidence gathered through the process of 
outcomes assessment. The final step in the assessment cycle entails 
using the evidence collected in the outcomes assessment process to 
make positive changes. Seventy-nine percent of participants report 
that they are adequately prepared to implement changes based on 
evidence gathered in the program review process. Based on these 
findings, the two greatest areas of need where deans and directors 
feel they could use the most support are in the processes of cre- 
ating program outcomes and then actually interpreting the data 
they later collect. About one-fourth of participants reported being 
uncomfortable with these two areas whereas a higher number felt 
comfortable actually gathering the data and implementing change 
once the data was interpreted. 

In addition to having the knowledge needed to effectively con- 
duct program assessment, another factor that might be influencing 
the adoption of program assessment is the attitude toward and 
perception of assessment. While 67 percent say that they would 
participate in outcomes assessment activities even if it was not 
required by their college or university, half (51%) of the partici- 
pants reported that they feel that administrators are focusing too 
much on outcomes assessment. While 72 percent of participants 
agree that outcomes assessment is a valuable component of the 
program improvement process, only 37 percent feel that it is a valu- 
able enough component of student learning that it should guide the 
majority of program changes. 

Some challenges to more fully implementing assessment as 
a part of the continuous improvement process within the hon- 
ors community include the perception of its effectiveness and 
time commitment. Specifically, some participants questioned the 
credibility of program assessment: 24 percent reported that they 
feel that outcomes assessment is not a true reflection of program 
effectiveness and therefore that it should not carry much weight 
in the program planning process. Another challenge to effectively 
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assessing and improving honors programs is how time-intensive 
the process can be. Twenty-five percent of participants reported 
that the time invested in developing and maintaining an assessment 
program is not worth the information gained. If honors adminis- 
trators are going to be able to meet the demands of college and 
university administration to prove their effectiveness, then it will 
become essential that future deans and directors understand how 
and to what extent program assessment can play a role in doing 
so. With nearly a quarter of the honors administrative population 
expressing doubt or hesitation toward the usefulness and worth of 
program assessment, there is still work to be done before a culture 
of continuous improvement exists within the honors community. 

While the majority of participants (over 70%) reported being 
comfortable with all areas of the program assessment process, hon- 
ors directors and deans could likely still use training in these areas. 
Despite the regularity that directors and deans are reporting that 
their programs are participating in discussions of outcomes assess- 
ment as well as reporting that they are prepared to interpret evidence 
and implement changes, only 31 percent say that outcomes assess- 
ment data are actually being used to guide the majority of program 
changes. This finding demonstrates that honors deans and directors 
are struggling to apply the skills they have to “close the loop” and 
effectively apply assessment practices for the process of continuous 
improvement. Additionally, only 22 percent of participants agreed 
that the skills they received in their graduate or disciplinary train- 
ing effectively prepared them to administer outcomes assessment. 
In other words, administrators are having to learn these skills on 
the job. 

The honors community, much like higher education as a whole, 
still faces a number of challenges before program assessment 
becomes a part of the culture. The national honors community 
needs to focus on developing the skills to conduct program assess- 
ment and use the findings strategically to develop goals for future 
improvement if it is going to encourage greater use of assessment. 
Only once we begin to adopt these practices will we then be pre- 
pared to quantify the value added by each of our programs. 
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So 


Proving the Value of Honors Education: 
The Right Data and the Right Messaging 


BETTE L. BOTTOMS AND STACIE L. MCCLOUD 
THE UNIVERSITY OF ILLINOIS AT CHICAGO 


dministered within over 1,500 honors colleges and programs 

in two- and four-year institutions worldwide (National Col- 
legiate Honors Council (NCHC) 2017; Scott and Smith 2016; 
Wolfensberger 2015), honors education serves the best interests of 
students and adds quality to the academic mission of host institu- 
tions by promoting the highest intellectual standards. Necessarily 
differing in form and content, all honors programs and colleges 
share the goals of identifying and supporting the most talented 
students as they achieve success in college and as they learn how 
to prepare not only for successful careers, but also for lifelong 
learning and meaningful civic engagement (Humphrey 2008). Cer- 
tainly honors enthusiasts believe that these goals are met through 
innovative and challenging programming in areas of curriculum, 
undergraduate research, community engagement and service, and 
leadership. 

These beliefs, however, need to be backed by empirical data. Do 
honors programs and colleges achieve their goals? Do they increase 
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the success of their students? Do they add measurable value to their 
institutions? How do we know? What data are needed to prove the 
worth of honors education, and how should those data be commu- 
nicated to the administrators responsible for funding it—provosts, 
chancellors, and presidents? What are the obstacles to honors pro- 
grams and colleges’ ability to gather those data and persuade various 
audiences? Nationally, a growing body of evidence confirms that 
honors students are more successful than other students (e.g., Cos- 
grove 2004; Pritchard and Wilson 2003). That every specific honors 
college or program know—not just hope or think—that it is effec- 
tive in terms of recruiting, retaining, and promoting the success 
of its exceptional students is essential. Achieving this knowledge 
requires the right data, the right analyses, and the right commu- 
nication. This paper details several ways to accomplish this task as 
well as some of the obstacles to this effort. We approach the idea of 
assessment and evaluation—or more simply, documenting positive 
effects of programs and persuading others of those effects—with 
social psychological research methods and while considering the 
politics of today’s higher education landscape. Specifically, we dis- 
cuss how to obtain, understand, and use the simplest to the most 
complex data to prove the ultimate value of an honors program, 
and how to tailor messaging about those data. Honors colleges and 
programs are the model for undergraduate recruitment and suc- 
cess. Our goal is to help readers prove it. 


FROM THE SIMPLEST TO THE MOST COMPLEX DATA 


At least three things are necessary to make a compelling case 
that honors education is worth institutional investment: the right 
data, the right analyses, and the right communication of those data. 
Honors deans and directors must know their audience and adjust 
the message appropriately. Sometimes the simplest data and the sim- 
plest analyses are sufficient, especially if the audience already values 
honors education. Sometimes more complex data and analyses are 
necessary because deans and directors may encounter skeptics 
about the worth of honors. Moreover, because universities and col- 
leges today are often underfunded, administrators are constantly 
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looking for ways to scale back operations. Those hard decisions 
should be, but often are not, based on data illustrating whether 
programs benefit students. Complex data can prove the worth of 
the program, but they are worthless unless conveyed clearly and 
understood by the audience. An honors dean or director may only 
have an elevator ride to convince someone of the importance of a 
college's or program's worth. Impressions are formed quickly and 
are long lived (Fisk, Gilbert, and Lindzey 2010). 

To obtain and use simple and complex data effectively, hon- 
ors administrators must first choose the outcomes (the dependent 
variables, in methodological terms) that are to be measured—those 
outcomes that are most important to an institution. Of all the won- 
derful things an honors college or program does for a university, 
usually the most important ones to the financial bottom line (ie., 
increasing tuition) are recruitment and retention: attracting the 
best students to the campus and retaining them until they gradu- 
ate. Next, we share a few examples of how to provide evidence of 
such value, going from the simplest to the most complex evidence. 


WHAT IS THE RECRUITMENT VALUE OF HONORS? 


The first example is simple yet exceptionally effective in many 
situations. What is the recruitment value of honors? At the Univer- 
sity of Illinois at Chicago (UIC), the dean (the first author of this 
essay) found that a particularly effective data point was that “65 
percent of freshmen said they ‘would not have come to the Uni- 
versity of Illinois at Chicago had it not been for the UIC Honors 
College.” This data point was self-reported by students using a 
poll of the entering freshman class with only one survey question: 
“Would you have come to UIC had it not been for the Honors Col- 
lege?” Fully 65 percent of all honors students and 75 percent of our 
most prestigious diversity scholarship students said “no.” 

Of course, people are not always accurate in their self-reports 
(Azar 1997), but they certainly can be, especially when they remem- 
ber what they are being asked and when they have no motivation 
to lie about it. This survey item meets those criteria. Importantly, 
this piece of data costs nothing to obtain, is easy and quick to 
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communicate, and is persuasive. It is elegant in its simplicity. And 
it is music to the ears of an admissions director, provost, and presi- 
dent, each of whom is interested in supporting enterprises that 
increase recruitment, especially of top achievers, thereby bring- 
ing more tuition money to the university and relieving some of the 
financial stresses that most institutions suffer. 


WHAT IS THE VALUE OF HONORS EDUCATION IN SUPPORTING 
STUDENT SUCCESS? 


The next examples consider a different yet also crucial ques- 
tion: What is the value of honors education in supporting student 
success? Setting aside distal measures such as lifelong success indi- 
cators, which are exceedingly difficult and expensive to collect, the 
most important proximal measure of the impact of honors on stu- 
dent success is whether students graduate. We present three ways to 
address this question of the impact of honors education on gradu- 
ation rates. The first and second approaches reflect the standard of 
“elegant simplicity” while the third provides a similar message but is 
far more complicated, far more difficult to convey, yet far superior if 
the audience really cares about and understands data and statistical 
analyses. Offering evidence to an institution’s administration that 
honors helps retain and graduate students is important because 
administrators usually care about students being successful at their 
institutions and because retaining students also generates tuition 
revenue and affects the financial bottom line. Further, we believe 
that all honors programs and colleges should be performing these 
analyses regularly, not only to prove their effectiveness to others, 
but also to assess for themselves whether they are effective. 


Example 1: 
A Simple Comparison 


Question: Do honors students graduate at higher rates than 
other students on campus? Answer: Yes, of course. At the UIC 
Honors College, where our students are 40 percent Pell-eligible 
and so diverse that there is no racial majority, the graduation rate 
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was recently 88 percent, while the campus average of non-honors 
students approached 60 percent. These two simple data points are 
impressive, and for many purposes, such as talking with prospec- 
tive parents, they are enough. For other purposes, however, these 
numbers are not sufficient because they are not definitive about 
the impact of honors education specifically. Skeptics can claim that 
honors students are more successful (e.g., more likely to graduate) 
for reasons other than their honors experiences; they maintain these 
students are smarter to start with, which people often wrongly think 
is measured by standardized test scores; better prepared; richer; not 
first-generation college students; and a dozen other qualities that 
are stereotypical, although not always true, about honors students. 

Allowing people to believe these notions is problematic. They 
are claiming that honors students would have been just as success- 
ful even if they had not been in the honors college or program. 
If that were true, honors education would not be needed. Experi- 
enced honors administrators know this claim is not true—if only in 
our gut. It is not merely what students bring with them to an honors 
program that determines their greater success; it is what honors 
education does for them once they get there. Thus, the dean's or 
director's job is to prove that honors education has an effect above 
and beyond various individual students’ entering characteristics. 
Examples 2 and 3 consider how to make that case. 


Example 2: 
Data that Begin to Account for Students’ 
Entering Characteristics 


In Figure 1, we provide a simple way to illustrate the increased 
success of honors students while also accounting for alternative 
reasons for this success. These data are from a recent cohort of Uni- 
versity of Illinois at Chicago Honors College students. This graph 
shows an outcome (in this case, graduation rates) as a function of 
whether students are in the honors college, but further, also as a 
function of a third variable that is often claimed to account for the 
increased graduation rate of honors students: ACT score. 
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Specifically, five-year graduation rates (percentages of students 
entering at the same time) are graphed on the vertical (i.e., Y) axis, 
and groupings of ACT scores are graphed across the horizontal 
(ie., X) axis. The top gray line shows the graduation rate of honors 
college students; the bottom solid line represents all other students 
(non-honors college students) at the university in that cohort. The 
main effect, statistically speaking, of honors is clear, with that top 
gray line being 20-25 percentage points higher than the bottom 
solid line. But most importantly, that difference pretty much holds 
steady across each level of ACT score, down to around 21 or 22 
ACT points. In other words, the effect of honors education on grad- 
uation rates is evident regardless of entering ACT scores. 


Ficure 1. Five-YEAR GRADUATION Rates AS A FUNCTION oF ACT: 
UIC Honors vs. Non-Honors STUDENTS 
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Note: n indicates the number of students in the honors college sample. 
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One could similarly plot graduation rates across various other 
alternative explanations one by one, such as the number of enter- 
ing AP credits or high school GPA. If such a graph is explained 
clearly and simply, anyone can readily understand it regardless of 
statistical expertise; the graph makes it possible to see the effect of 
honors. It is therefore effective across audiences with widely vary- 
ing levels of statistical sophistication. For example, we used it in our 
annual report, which is aimed at administrators, faculty, students, 
alumni, donors, and other friends (Bottoms, Mehta, and McCloud 
[Williams] 2015). That gap between the gray and solid lines in that 
graph represents the “value added” of the honors college and clearly 
illustrates that, again, what is consequential is not what students 
come with when they enter college but what honors does for them 
once they arrive. 

Another point worth noting in Figure 1 is the 100 percent 
graduation rate for honors students with ACT scores of 21 and 
22. These scores are not often seen among honors students, given 
typical admissions policies. Although only eight honors students 
had scores in this range, their success demonstrates better than any 
other group the value that honors adds. Moreover, students at the 
top of the ACT distribution—even those students with ACT scores 
of 35 and 36—are no more likely to graduate than those with much 
lower scores, and non-honors students with 35 or 36 are certainly 
less likely to graduate than the honors students with the lowest 
ACT scores. Considering how flat both the gray and solid lines are 
is important: ACT score—above about 21 or 22—is not a strong 
predictor for anyone at UIC, which can be seen here because hon- 
ors and non-honors student groups have been pulled apart, that is, 
separated. Note that if this graph had merged the two groups into 
only a single line, one would have seen a slight upward slope from 
left to right, a trend that would also show up as a small statistical 
correlation if one did the calculations. But the graph in Figure 1 
illustrates that such an association of ACT with graduation rates is 
due to honors college membership—simply more of the honors col- 
lege students have higher ACT scores, and the honors students also 
graduate at higher rates. That is, higher ACT scores are somewhat 
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confounded with honors college membership, so when honors stu- 
dents are pulled out, the line flattens, and the correlations drop to 
non-significance, at least when the lowest ACT scores in the analy- 
sis start around 21 or higher. Administrators at universities often 
wrongly use such a one-line approach, without disentangling hon- 
ors and non-honors students, in arguing that ACT scores predict 
student success. This graph, therefore, underscores the importance 
of the growing and well-supported movement to admit students 
based on factors other than standardized test scores. 


Example 3: 

Complex Data, Complex Analyses: 

A Comparison that Accounts Well for Students’ 
Entering Characteristics 


The data presented in Figure 1 provide an important illustra- 
tion for administrative audiences, but ultimately, it is still not a 
completely definitive answer to claims of alternative explanations 
because it considers only one alternative explanation at a time, such 
as ACT, and because it is not a statistical analysis that can provide a 
more specific estimation of effect sizes. Statistically speaking, vari- 
ous predictor variables, such as ACT, high school GPA, or whether 
a student is first generation, can be interrelated with each other, so 
one needs to look at all of them simultaneously to understand the 
unique effects of each and to understand whether honors educa- 
tion has an effect above and beyond all those other factors. A more 
sophisticated approach that takes care of these concerns is to use 
multivariate statistical analyses, which control for many variables at 
once to see the unique effect of the honors experience. 

Researchers at UIC have done such analyses, and this study is 
presented in detail in another chapter of this collection (see Diaz, 
Farruggia, Wellman, and Bottoms (2019) herein). To summarize 
briefly, we studied over 21,000 students who entered UIC between 
2006-2012, 14 percent of whom were honors college students. The 
sample was unusually diverse, as is our institution (US. News & 
World Report 2017), with a mean age of 18 years; 55 percent women; 
37 percent first-generation college students; 45 percent Pell-eligible; 
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and 24 percent Asian, 10 percent African American, 21 percent 
Latino, 35 percent white, and 10 percent mixed or other ethnicities/ 
races. We conducted five separate analyses for five outcome mea- 
sures: hierarchical linear regression models were used to assess the 
effects of honors college membership and other variables on first- 
semester GPA and number of credits completed in the first year 
of college, and logistic regression was used to assess the outcomes 
of retention from the first to second year, 4-year graduation rate, 
and 6-year graduation rate. In addition to determining the impact 
of students’ membership in the honors college, we tested for the 
potential effects of nine additional predictor variables that might be 
confounded with honors college membership, and which therefore 
could be alternative explanations for the effect of honors. Specifically, 
in each of the five analyses, all predictor variables were entered into 
six steps or “blocks” as follows: (1) age and gender; (2) ethnic/racial 
background; (3) parent income and first-generation status; (4) enter- 
ing high school GPA, number of AP credits, ACT composite score, 
and UIC writing placement; and finally (5) honors college partici- 
pation. Readers need not understand statistics deeply to appreciate 
the basic idea of how these analyses work. Essentially, these analyses 
detect and pull out the statistically significant (ie., reliably detect- 
able) effect of one variable after another, until all variables have been 
accounted for. In other words, the first step (block) of one of these 
analyses first accounts for (or pulls out) whatever statistical impact 
age and gender might have. Then the next steps account for any effect 
that race/ethnicity has, and so on until the only variable left is honors 
college membership in the last step. If the effect of honors college 
membership were due to its being confounded with any or all of the 
other variables, then logically, it would have no statistically signifi- 
cant effect when added in step 5, because at the end of the analysis, 
the effects of all the other variables have already been accounted for. 
If honors college membership still has a significant effect in step 5, 
then that effect is really due to honors and not to any of the other 
variables that have already been accounted for. 

As detailed later in this volume, Diaz et al. found that com- 
pared to non-honors students, honors college students had higher 
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first-term GPAs, earned more credits their first year, were more 
likely to be retained to their sophomore year, and had higher four- 
and six-year graduation rates. Importantly, those effects remained 
even after the analyses controlled for the effects of all the other 
nine alternative variables: honors college membership still had a 
significant effect in step 5. Therefore, the analyses illustrated the 
significant impact of honors college membership above and beyond 
the effects associated with nine other variables that are often con- 
founded with honors college membership, thus ruling out many 
alternative hypotheses that are often used by critics to explain away 
the positive effects of honors education. 

Further, our analyses also indicate that the benefits of honors 
college membership increase with the amount of time students 
spent in the honors college. This phenomenon argues against 
another alternative explanation for the impact of honors educa- 
tion—that honors students self-select because of higher initial 
internalized motivation to succeed and that this motivation rather 
than their experiences in honors leads to their higher levels of 
success. Another argument against this alternative motivational 
explanation is that high school grades are surely, at least in part, a 
simple partial proxy for motivation, and we also controlled for that 
and still found the effects of honors to be significant. 

Finally, our analyses also revealed another important factor: 
African American and Latino students benefited more than did 
students in other racial/ethnic categories, at least in terms of first- 
term GPA and first-year credits earned. Documenting that honors 
can play a role in decreasing the huge gap in educational achieve- 
ment between underrepresented minority students and others is 
important. Being able to present such evidence is truly gratifying. 
Honors colleges and programs can and should admit more prom- 
ising underrepresented minority students, de-emphasizing factors 
such as standardized test scores. When we took this step at UIC, 
skeptics opined that we were only setting students up for failure. 
We were not. Our analyses provide evidence that honors supports 
their success. 
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LIMITATIONS, POTENTIAL BARRIERS, AND 
OTHER CONSIDERATIONS 


Finally, it is worth considering potential limitations and obsta- 
cles to the approaches we have discussed here in order to be pre- 
pared to address them if they are mentioned by the audience one is 
trying to persuade. First, no single analysis is perfect, and the best 
strategy for explaining a complex human behavior such as college 
student success is to have a multifaceted plan that builds a case on 
the basis of converging evidence. Even in our multivariate regres- 
sion analyses, we certainly did not test every possible variable that 
could be confounded with honors college membership; thus more 
work can be done to identify and test other alternative hypotheses. 
And of course, all of our data come from students at one univer- 
sity. We have presented several types of converging data, but many 
other possibilities exist depending on the particular program and 
on what outcomes and predictors are important at the institution. 

Second, even though the analyses support the contention that 
honors education is effective, they do little to explain why. Hon- 
ors programs are home to many academically enriching (i.e., 
“high-impact practices” a la Kuh 2008; Mayhew et al. 2016) and 
socio-emotionally supportive programs, including specialized pro- 
fessional, peer, and faculty advising; engaged living communities; 
financial assistance; special academic work such as small honors 
classes, capstone theses, and research; leadership experiences; and 
service learning and civic engagement experiences. All of them 
have been linked in general student populations to increased college 
success (e.g., Inkelas and Weisman 2003; Freeman et al. 2007). But 
questions remain: which among those programs are most effective 
at improving student outcomes? Which ones contribute most to the 
positive effects of honors education? Investigation of these factors 
is not only important for supporting lobbying efforts on behalf of 
an honors college or program with university administration, but 
also for program evaluation and development purposes, for under- 
standing where to focus resources, and for staff training. Further, it 
is important to identify which practices are best for which students. 
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This information could lead to understanding why the effects of 
honors experiences are stronger for students of some races/ethnici- 
ties compared to others. A tailored approach to honors education, 
heeding individual differences in student needs, fits with the val- 
ues of honors. Such analyses would also provide information about 
practices that help all students, not only honors students—again, a 
goal that fits well with the values of honors. 

Third, we discussed only relatively short-term outcome mea- 
sures, such as credits earned during the first year, college GPA, 
retention, and graduation. These measures are certainly important, 
yet the literature on program evaluation makes it clear that many 
ways to determine effectiveness exist. Higher education, espe- 
cially honors education, claims to prepare students for a better life 
beyond college. Does it? Ideally, honors programs would track their 
graduates to obtain richer long-term measures of success, including 
evidence of lifelong learning and being responsible in civic society. 

Fourth, one barrier to programs or colleges collecting the type 
of data and doing the kinds of analyses suggested here might be 
that honors administrators, especially deans and associate deans, 
while possessing expertise in a broad range of areas, often come 
from disciplines unfamiliar with multivariate statistical techniques. 
If they lack these skills, one solution that will work is to request 
that the office of institutional research perform the analyses. Of 
course, institutional research and reporting staff members may 
not have time to fulfill individualized department requests, or they 
may specialize in purely descriptive analyses rather than social sci- 
ence hypothesis testing and analyses using multivariate regression 
or hierarchical linear modeling (HLM). If that is the case, another 
possibility is engaging successful faculty members from the social 
sciences to lead these efforts. Faculty who have published papers 
using these analyses to examine human behavior or highly quali- 
fied graduate students under their supervision can conduct and 
explain the analyses. These researchers should be encouraged to 
capitalize on their need to publish by allowing them to use the data 
for testing theories that interest them, and they should be compen- 
sated appropriately whenever possible. 
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One caveat to remember when selecting faculty partners is that 
a little statistical knowledge is a dangerous thing. There are many 
ways to conduct technically legitimate analyses, especially regres- 
sions, but without expertise in using and interpreting such analyses, 
one can end up with an inaccurate story of human behavior. People 
who are new to statistics or use them infrequently might not under- 
stand how to answer various questions using the proper analysis or 
the proper statistical controls. Consequently, relying on truly expe- 
rienced faculty partners is advisable. Finally, just because handling 
data statistically might be unfamiliar to an honors administrator, 
that is no reason to fear it or accompanying tasks such as having a 
plan reviewed by the Institutional Review Board. The right faculty 
partners will be well-versed in how to present studies for human 
subjects review, and the study could even move forward with what 
is known as “exempt” status if it is done with appropriate safeguards 
for the confidentiality and anonymity of student participants. 

Fifth, funding may not be available to support this work. Because 
of a tight budget, honors administrators might decide that using 
resources to support programs rather than investing in evaluative 
data collection and analyses is a better choice. We challenge that 
assumption. If administrators do not know whether their programs 
are working, continuing them might not make sense. It is essen- 
tial, then, to recognize the importance of investing in data analysis, 
understanding that the day will come—if it has not already—when 
an administrator above the honors college or program will demand 
good evidence before continuing funding and institutional sup- 
port. Before that day arrives, honors deans and directors should 
designate or redirect program funds or look for alternative sources 
of support such as grants to conduct these studies. We would also 
urge honors administrators to be creative: many private founda- 
tions have an interest in higher education, especially in research 
that can generalize nationally, and some foundations and even 
individual donors may specifically be interested in high-achieving 
students, or else we would not have examples such as the Lewis 
Honors College, the Schreyer Honors College, or the Barrett Hon- 
ors College. The campus office of development or advancement can 
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help honors administrators identify foundations and people who 
care about such causes. We also suggest that honors administra- 
tors “Google” around on their own to find possible contacts. Also, 
businesses invest in efforts that have the potential to enrich their 
pipeline of employees, so we advise honors administrators to make 
connections in their locale. Honors directors and deans can lobby 
honors organizations such as NCHC to create special small grants 
for this purpose, and, as mentioned above, it is always possible to 
engage faculty and graduate students who might be willing to work 
for the benefit of potential publication alone. 

Finally, perhaps the most depressing potential possibility is that 
no one will listen to or believe the honors administrator, even when 
the right data are presented, analyzed properly, and communicated 
correctly. Some opponents to honors, especially those with a danger- 
ously small amount of statistical knowledge will pick at everything 
an honors administrator does—no matter what variables one tests 
or how many tests are run. No matter how conclusive the analysis 
is, it may never be enough to convince some for whom there will 
always be an alternative explanation. To that end, assembling con- 
verging data is essential. In addition to the kinds of data we have 
discussed, honors administrators should add qualitative or descrip- 
tive information and case studies with narratives about students 
who have succeeded because of honors and despite academic false 
starts. Being thorough and persistent is critical because people may 
reject the results outright for no good reason. If social psychologists 
know anything, it is that people believe what they want to believe 
(i.e., the “confirmation bias,’ e.g., Kassin, Dror, and Kukucka 2013; 
Rosenthal and Jacobson 1966), and leaders in higher education are 
no exception. The first author once presented data to an interim 
provost who was so dismissive that he only smiled condescendingly 
and said he would “have to wait to see the publication.” Many com- 
peting political interests flourish at a university, but not all of them 
are admirable. Certain special interests may trump good data and 
best intentions. That does not mean, however, that honors adminis- 
trators should not do the research for their own internal evaluation 
needs or for discussion with high-level academic administrators 
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when regime change occurs at high levels of the institution's admin- 
istration, as there is fairly often these days. Moreover, producing 
such research can add to the accumulating evidence in the grow- 
ing publication record about the impact of honors education. Our 
advice is that honors administrators should steel themselves, be 
patient, do the right thing, collect good data, analyze those data cor- 
rectly and honestly, explain the analyses well, use the right data for 
the right situation, publish it if possible, and above all, be persistent. 


CONCLUSION 


Nationally, the appreciation of multivariate social science sta- 
tistical methods to investigate many aspects of higher education is 
increasing. Honors should be no exception. In turn, such analyses 
will increase the quality of honors education, especially if leaders 
have the courage to share their results and act on them appropri- 
ately. Some efforts are already underway, such as this monograph, 
which follows from fruitful discussion among many concerned 
leaders in honors education who met at the May 2016 NCHC- 
sponsored honors research colloquium at Wayne State University, 
organized by NCHC Past President Jerry Herron to further his ini- 
tiative as president to emphasize honors research. Another example 
is the establishment of Honors Education in Research Universities 
(HERU), a collective with the goal of “fostering the extension of the 
unique research mission of our institutions to our honors colleges 
and programs . . . to truly understand the efficacy of our efforts”; 
and the creation of HERU’s new online open-access journal, Hon- 
ors in Higher Education, to “foster creative thought about how to 
achieve a more sophisticated level of self-examination through 
research” (Bottoms and Gutgold 2016). More research is needed, 
and we see many indicators that leaders in honors are rising to the 
occasion. 

Especially in fiscally tight times, it is important for any honors 
college or program to prove that it is effective. The right data, the 
right analyses, and the right communications will reveal unequivo- 
cally that honors education is an effective model for undergraduate 
recruitment and success. 
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QO 1,500 U.S. universities and colleges have honors pro- 
grams or honors colleges to provide extra support for their 
most prepared students (National Collegiate Honors Council 2018; 
Scott and Smith 2016). Honors programs typically provide addi- 
tional financial support, faculty mentors, smaller class sizes, and 
other benefits compared to what institutions can typically offer all 
of their students. Students involved in an honors program usually 
earn higher GPAs compared to highly motivated students not in an 
honors program (Pritchard and Wilson 2003) and are more likely 
to stay in college and graduate within four years (Cosgrove 2004). 
The additional success of honors students compared to non- 
honors students is often attributed to their experiences in the 
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honors program itself. But it could be argued that honors students 
are more successful simply because they arrived at a university with 
better preparation or higher socioeconomic status. Of course, no 
explanation can be definitive without a randomized control trial, 
which would be difficult if not impossible in real-world situa- 
tions, but converging evidence from multiple sources can provide 
a reasonable answer (Bottoms and McCloud 2018). Consider- 
able research to date on the impact of honors education lacks the 
appropriate controls to account for alternative explanations for the 
differences often observed in the success of honors versus non- 
honors students. The present study tests the impact of an honors 
college on the successes of a diverse, urban student sample while 
statistically accounting for pre-matriculation background factors 
and student characteristics, thereby ruling out many key alterna- 
tive explanations for the association between honors education and 
college student success. 


PRIOR RESEARCH ON THE IMPACT OF HONORS EXPERIENCES 


Many researchers have found a positive association between 
honors colleges and college success. For example, Hébert and 
McBee (2007) found that honors programs and the community 
they create allowed students to become involved in more than 
academics and to develop intrinsic motivation and self-efficacy. 
Castro-Johnson and Wang (2003) found that honors students had a 
higher first-year GPA and higher emotional intelligence scores than 
their non-honors peers, and they also had higher entering high 
school GPA and ACT scores. Cosgrove (2004) compared students 
who stayed in an honors program throughout their college career 
to (a) other high-achieving students not in the honors program and 
(b) students who started out in the program but failed to complete 
it. Those who stayed in the program had higher GPAs and shorter 
times to degree completion compared to both other groups. Keller 
and Lacy (2013) found that compared to similar non-honors stu- 
dents, students who participated in an honors program had higher 
rates of first-to-second-year retention, as well as higher four-, five-, 
and six-year graduation rates. Such studies are useful in forming 


60 


POSITIVE EFFECT 


a growing body of converging evidence illustrating the value of 
honors programs, but they are often limited by examining only one 
cohort, including samples that are not ethnically diverse, and not 
controlling for potentially confounded and explanatory factors. 
Why might honors programs promote student success? Honors 
colleges and programs might promote student success because they 
provide students with myriad supports across many domains: social, 
emotional, informational, financial, and academic. For example, 
Hébert and McBee’s (2007) qualitative study concluded that honors 
programs can provide students with intellectual and psychosocial 
growth, especially by providing faculty mentors. Another hallmark 
of the academic experience provided by most honors programs is 
the use of what Kuh (2008) referred to as “high-impact practices,’ 
including first-year seminars, learning communities, collaborative 
assignments and projects, problem-based learning, undergradu- 
ate research, service learning, and capstone courses or projects. 
(For reviews of work addressing such activities, see Kuh, Kinzie, 
Schuh, Whitt, and Associates 2010; Mayhew, Rockenbach, Bow- 
man, Seifert, Wolniak, Pascarella, and Terenzini 2016; McKay 
and Estrella 2008). High-impact practices lead to greater student 
retention and graduation rates because, compared to standard edu- 
cational practices, they engage students more in their college work 
and with faculty members, their peers, and their campus so that 
they feel a greater sense of academic and social belonging to their 
campus. Having a greater sense of belonging to the campus has 
been positively associated with personal motivation, perceptions 
of professors, and a greater sense of social acceptance (Freeman, 
Anderman, and Jensen 2007). For example, a typical centerpiece of 
honors education is undergraduate research experiences, especially 
a capstone project. Considered a high-impact practice, research 
increases student engagement with faculty members and peers on 
campus (Hartmann, Widner, and Carrick 2013; Kuh 2008) and is 
related to academic achievement (Webber, Laird, and BrckaLorenz 
2013). Students have reported satisfaction from connecting their 
research to real life, developing a community with other students, 
finding mentors among the faculty involved, and gaining owner- 
ship over their learning experiences (Falconer and Holcomb 2008). 
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RACIAL / ETHNIC DISPARITIES IN COLLEGE STUDENT SUCCESS 


Much research has examined racial/ethnic disparities in col- 
lege student success, especially the tendency for African American 
and Latino/a students to graduate at substantially lower rates com- 
pared to White and Asian American students (National Center for 
Education Statistics 2013). Underrepresented minority students are 
typically considered to be “at risk” (Schreiner, Noel, Anderson, and 
Cantwell 2011), risk that is often explained by background char- 
acteristics such as being the first in their family to attend college, 
coming from a lower socioeconomic status, or being an ethnic 
minority student enrolled in a predominantly White institution 
(e.g., Bryan and Simmons 2009; Walpole 2008; Zwick and Sklar 
2005). Compared to their White peers, underrepresented minority 
students might have lower social capital and less access to networks 
that can provide support for college students, and as a result they 
have less access to resources such as money and academic and 
socioemotional supports (Bastedo and Gumport 2003). 

Because of such disadvantages, participation in a supportive 
environment such as an honors program might provide Latino/a 
and African American students with support and resources that 
they do not have access to otherwise, perhaps even dispropor- 
tionately more so than White students. In fact, Seifert, Pascarella, 
Colangelo, and Assouline (2007) found that underrepresented 
students in an honors program scored higher on a reading compre- 
hension exam than those not involved in honors, while there was 
no difference in scores between the White students who were in 
the honors program versus those who were not. Honors programs 
might be particularly helpful for ethnic minority students who gen- 
erally report fewer and less satisfying interactions with faculty, both 
socially and academically. In fact, Inkelas and Weisman (2003) 
found that students within an honors program were more likely 
than non-honors students to discuss academic issues and concerns 
with faculty and peers. 

Further, positive faculty-student interaction is associated with 
successful academic performance (Anaya and Cole 2001) and 
varies according to minority status. McKay and Estrella (2008) 
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found that service learning helped first-generation students suc- 
ceed academically via greater engagement in course material with 
faculty members. Anaya and Cole (2001) noted that underrepre- 
sented minority students might be less likely to engage with faculty 
than their White counterparts, which could explain some of the 
disparity in their college success. Lundberg and Schreiner (2004) 
found that African American students reported lower levels of sat- 
isfaction with faculty relationships compared to White students. 
African American students and Native American students reported 
the most interaction with faculty, but they felt they had to push 
themselves harder than White students to meet faculty expecta- 
tions. They also reported less satisfying relationships with faculty. 
Thus students of varying racial/ethnic backgrounds perceive differ- 
ences in experiences with faculty, experiences that are important to 
college success. Again, because honors colleges and programs pro- 
mote high-quality faculty/student interaction, honors education 
might have even more influence on underrepresented minorities 
than on other students. 

As previously mentioned, increased honors student success 
might be due to stronger feelings of belonging on campus, something 
that might also be more important for underrepresented students 
than White students, again providing reason to expect honors edu- 
cation to provide even more benefit for underrepresented students. 
For example, Nora, Barlow, and Crisp (2006) found that one of the 
reasons why minority students were not retained is that they did not 
have a strong sense of belonging. Similarly, Lundberg and Schreiner 
(2004) found that for many African American students, dropping 
out of college was less related to GPA than to feeling isolated and 
not supported on campus. Kuh (2008) and others claim that for 
this sense of belonging to occur, students must feel that there is a 
“critical mass” of students like them on campus. One could argue 
that an honors program provides minority students with a differ- 
ent type of critical mass to identify with—close peers of a similar 
high-achieving mindset. In support, Fries-Britt (1998) found that 
African American students in a merit-based scholarship program 
did not feel a sense of belonging and community with non-honors 
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African American peers, feeling instead more connected to other 
high-achieving students, even those from different racial/ethnic 
backgrounds. The students reported believing that the non-honors 
African American community thought that program participants 
had special treatment over them, but the students in the program 
believed that they benefitted from having the additional resources 
provided to them, having a community of high-achieving African 
Americans to interact with and gain support from, and having fac- 
ulty with high expectations of them. Therefore, honors programs 
might isolate students from others not in similar programs, but 
they can also foster a sense of belonging and social support. 


STUDY RATIONALE AND HYPOTHESES 


Little research addresses ethnic/racial group differences in stu- 
dent success in the context of honors education (for a discussion, 
see Coleman, Kotinek, and Oda 2017). This gap is probably because 
most honors programs admit relatively few underrepresented stu- 
dents, although we know of no studies specifically documenting 
this situation. Studies that do include race and ethnicity rarely 
have a truly diverse population, often have a disproportionately 
high White demographic, describe their sample broadly as “White” 
and “non-White” (Singell and Tang 2012; Keller and Lacy 2013; 
Furtwengler 2015), or provide a detailed breakdown without pro- 
viding separate results for each group (Pritchard and Wilson 2003). 
In addition, most studies do not control for other student back- 
ground factors, such as high school performance, parent income, 
and parent education. For example, Furtwengler (2015) found that 
students who were typically less likely to enroll in an honors col- 
lege program (calculated using a propensity score) were those who 
benefitted most from the program in terms of higher GPA, yet this 
study failed to control for parent socioeconomic status and par- 
ent education, nor did it include measures of retention and time 
to graduation. Our study meets the need for more studies of ethni- 
cally diverse populations within honors and is especially important 
given the unique experiences and needs of ethnic minority students 
in universities. 
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To preview, we used statistical analyses to test the association 
of honors involvement with greater academic success once other 
potentially explanatory background variables were accounted for 
and to determine whether underrepresented minority students 
benefitted more from honors education than did other students. 
We hypothesized that when student background variables were 
accounted for, compared to non-honors students, students in a 
university honors program would (1) have higher first-term col- 
lege GPA, (2) earn more credits during the first year, (3) be more 
likely to persist from the first to second year, and (4) be more likely 
to graduate at four and six years after matriculation. (Most hon- 
ors students who graduate do so in four years; see Cosgrove 2004.) 
We included first-term GPA, first-year credits earned, and first- 
to-second-year retention as outcomes because success in the first 
year of college is an important predictor of graduation and success 
in college (Tinto 1993). In addition, we predicted that the effects 
for Latino/a and African American students compared to their 
Asian American and White peers would be larger. Finally, we also 
explored for differences in the associations between honors partici- 
pation and student outcomes based on the student’s point of entry 
into the honors program and how long students were in the honors 
program. That is, while many students began their time in honors 
programs during the first semester of their first year, some students 
at the university entered later after demonstrating academic success 
at the institution. (We did not include students who transferred into 
the university from other institutions.) We expected that more time 
in an honors program would lead to even more positive academic 
outcomes. 


THE RESEARCH CONTEXT 


We conducted research at the University of Illinois at Chicago 
(UIC), a large, Midwestern, public, urban research university with 
over 17,000 undergraduate students and a well-established honors 
college. Although students may apply to this honors college any 
time before their penultimate semester in college, most enter as 
first-year students. Students in the honors college are selected in a 
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holistic manner that considers background characteristics such as 
high school grades, record of civic engagement and other extra- and 
co-curricular activities, various aspects of verbal and interpersonal 
performance during an in-person interview, the quality of essays 
written at two different times, diversity considerations broadly 
defined and consistent with considerations laid out in Grutter v. 
Bollinger (2003) and upheld in later decisions such as Fisher v. Uni- 
versity of Texas (2016), and to a lesser extent ACT scores. Any high 
school seniors may apply to the honors college as a part of their 
college application or later in early spring before university matric- 
ulation. Those who have reasonably strong academic backgrounds 
(usually high school grades of B or higher, but not necessarily high 
ACT scores) are invited to participate in an in-person interview, 
which is conducted and assessed by trained interviewers, and to 
complete written essays to allow for the assessment of other criteria. 

This honors college provides a host of supportive experiences 
for students, including high-impact academic practices such as two 
required small, interactive, honors-only, three-credit general-edu- 
cation first-year seminars, which include field trips, projects, and 
papers that engage students with each other and their professor. 
(For details, see Bottoms, Mehta, and McCloud [Williams] 2015.) 
Honors college first-year students also take one-credit first-year- 
experience seminars that prepare students to take advantage of 
what the university and honors college offer, facilitated by a peer 
mentor and often taught by the students’ professional honors 
advisor (Chang, Hall, and Bottoms 2016). This advisor provides 
academic, informational, and socioemotional support throughout 
the students’ years in college. Co-curricular and extracurricular 
activities with academic components are also a necessary aspect of 
membership in the honors college, with 45 hours of honors activity 
being required of the students each semester. These activities range 
broadly given student interest, and they include student organiza- 
tion leadership, community service learning projects, one-credit 
advanced honors seminars, study abroad, extra projects contracted 
in existing courses, research and other creative independent stud- 
ies, and internships. All honors college students are also required 
to participate in research or other comparable scholarship in their 
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discipline, including an independent senior capstone project with 
faculty oversight. Honors college students also receive extra aca- 
demic and disciplinary advising and mentoring from an assigned 
honors college Faculty Fellow, with whom they meet at least twice 
per semester for academic and career guidance and support. The 
honors college also provides myriad other supports, such as hosting 
lectures and activities where students and faculty interact, field trips 
to major cultural events in the city, and access to special facilities 
such as computer and study rooms and living-learning commu- 
nities in the residence halls. The curriculum and programs often 
include specific attention to diversity, broadly defined, reflecting 
the nature of the college's unusually diverse student body (Chang 
et al. 2016). 

Previous research at this university on the impact of pre- 
matriculation characteristics on student success has demonstrated 
that high school grades, Advanced Placement (AP) credits earned, 
and race/ethnicity have consistent and significant associations with 
grades, retention, and graduation (Farruggia, Bottoms, Leighton, 
Wellman, and Moss 2016; Farruggia, Han, Watson, Moss, and Bot- 
toms 2016). Although with very small effects and not consistent 
across all outcomes, gender, age, parent education, parent income, 
ACT score, and placement were also sometimes associated with 
student success at this university. 


METHOD 
Participants 


The sample comprised all full-time first-year students who 
entered the university in the fall terms between 2006 and 2012 
(inclusive) (N = 21,723). The group (55% female, M age = 18 years, 
SD = .79) was ethnically diverse (35% white, 24% Asian American, 
21% Latino/a, 10% African American, and 10% other) and socioeco- 
nomically diverse (37% first-generation college students; 45 percent 
eligible to receive Federal Pell Grant funding; parental income M = 
$67,037). Fourteen percent of the students were in the honors col- 
lege for at least one term during their time at the university. 
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Measures and Procedure 


All data were archival, downloaded from the university data 
warehouse in keeping with an approved university Institutional 
Review Board protocol and consent from the Provost, Director of 
Financial Aid, and the Vice Chancellor for Students. Student back- 
ground data included gender, race/ethnicity, and age (in years). 
Parent background data included parent income in dollars and par- 
ent education, with the latter used to code students as being first 
generation in college = 1 (neither parent having graduated from 
college) or not first generation in college = 0 (one or both parents 
had graduated from college). 

Pre-college-matriculation data (i.e., high school achievement 
data) included students’ high school GPA, number of AP credits 
earned, ACT Composite scores, and writing-course placement 
scores. High school GPA was unweighted and measured on a 
4-point scale (where 4 was highest). Number of AP credits earned 
reflected the total number of UIC credits awarded to a student 
based on AP tests taken in high school, as well as dual enrollment 
credits earned (which were rare). Writing-course placement scores 
came from either (a) placement exams taken the summer before 
the first semester in college or (b) on the basis of automatic place- 
ments based on AP test scores, ACT/SAT scores, or community 
college credits. A score of 1 indicated that the student was assigned 
to the most introductory, non-credit-bearing writing course level, 
and a score of 5 was the most advanced writing course level. Data 
were largely complete (99%+) for all of these indicators. 

We measured honors college participation in three ways: (1) 
dichotomously: whether the student was in the honors college start- 
ing the first semester of the first year in college (yes = 1, no = 0); (2) 
as a ratio term: the sum of the number of semesters the student was 
in the honors college divided by the total number of semesters at 
UIC; and (3) dichotomously: whether a student was ever in the UIC 
Honors College during any semester (yes = 1, no = 0). (First-year 
students could enter the honors college after the first semester.) 

We measured success in terms of first-semester GPA, credits 
earned in the first year, first-to-second-year retention, four-year 
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college graduation, and six-year college graduation for the first 
measure of honors participation. For the ratio term and whether 
the student was ever in the honors college, we measured success 
in terms of four-year college graduation and six-year college par- 
ticipation. First-term GPA was based on a 4.0 scale and was the 
average of grades earned in all credit-bearing courses in the first 
semester as calculated by the university. Credits earned in the first 
year reflected the cumulative credits earned by the end of the first 
year, including summer session if taken prior to freshman year. 
First-to-second-year retention, four-year graduation, and six-year 
graduation were dichotomous variables indicating whether the stu- 
dent was retained or had graduated (yes = 1, no = 0). 


Detailed Plan of Analysis 


Independent samples t-tests and chi-square analyses tested 
for significant differences between honors and non-honors college 
students in terms of gender, age, race/ethnicity, parent education, 
parent income, high school GPA, AP credits earned, ACT Compos- 
ite score, writing placement test, first-term GPA, credits earned in 
the first year, first-to-second-year retention, four-year graduation, 
and six-year graduation. We used logistic regression to examine dif- 
ferences between honors college and non-honors college students 
simultaneously with student and family background to determine 
if some of these were no longer significant when examined simul- 
taneously. We coded race/ethnicity using dummy codes where 
Asian American students were the reference group. To determine 
if participation in the honors college was associated with greater 
student success, hierarchical linear regression and logistic regres- 
sion analyses examined success outcome variables. This approach 
allows for controlling of background variables to understand the 
unique contribution of honors college participation above and 
beyond effects that might be associated with other variables. Spe- 
cifically, for first-term GPA and number of credits earned, we used 
hierarchical regression, controlling for background character- 
istics. Honors college participation was measured in this model 
by whether the student was in the honors college his or her first 
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semester. Predictors were entered into six blocks: block 1, age and 
gender; block 2, racial/ethnic background; block 3, parent income 
and whether or not students were first-generation students; block 
4, high school background characteristics (high school GPA, num- 
ber of AP credits, ACT Composite scores, and writing placement 
scores); and block 5, honors college participation in the first semes- 
ter. We separated race/ethnicity from age and gender into different 
blocks so we could clearly see if race/ethnicity had a direct effect 
on the outcome variables. We performed independent regres- 
sions for the separate outcomes of first-term GPA and number of 
credits earned in the first year. For the dichotomously measured 
outcomes of first-to-second-year retention and four- and six-year 
graduation, we performed separate logistic regression analyses, but 
otherwise the models were similar, with background characteristics 
controlled. 

Given that Latino/a and African American students typically 
have lower rates of success compared to Asian American and white 
students, we were interested in determining whether honors college 
education would help to close the achievement gap. To determine 
if honors college participation had a greater effect on Latino/a and 
African American students, as mentioned previously, we created 
dummy codes with Asian American students as the reference group 
because they had the highest overall success of the four racial/eth- 
nic groups in the general student population. Then, we created three 
interaction terms by multiplying the race/ethnicity dummy code by 
the honor college participation variable. Similar regression analy- 
ses were conducted a second time to add these interaction terms 
(in block 6). By keeping these interaction terms separate from both 
race/ethnicity and honors college participation variables, the analy- 
sis could test whether those additional variables significantly added 
to the model. 

To examine if more time in the honors college was associated 
with better student success outcomes, which was only relevant for 
the dependent measures of 4- and 6-year graduation, we calculated 
a ratio variable: the sum of the number of semesters that the student 
was in the honors college divided by the total number of semesters 
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that the student was at UIC. We created interaction terms with 
this ratio variable and the race/ethnicity dummy variables. Regres- 
sions tested models similar to those just explained except that (1) 
the ratio variable replaced the honors college participation variable 
in block 5, (2) the corresponding interaction terms replaced prior 
interaction terms in block 6, and (3) the models were used to pre- 
dict only the dependent variables of four- and six-year graduation. 

A third set of regression analyses using similar models with new 
dependent measures tested whether participation at any time in the 
honors college was associated with better student outcomes as mea- 
sured by 4- and 6-year graduation. We created interaction terms 
that crossed any honors college attendance with the race/ethnic- 
ity dummy variables. Finally, we conducted additional regression 
analyses using models similar to those previously explained except 
that the honors college participation variable was replaced in block 
5 and corresponding interaction terms were entered in block 6. 


RESULTS 
Preliminary Analyses 


Preliminary analyses considering direct relations between 
honors college membership and each variable separately (without 
simultaneously controlling for other variables) provided zero-order 
relations and informed our choice of variables to use as controls 
in our main model-testing analyses presented below. Specifically, 
a series of y’ tests and t-tests revealed statistically significant differ- 
ences between honors college and non-honors college students for 
gender, age, race/ethnicity, first-generation college students, par- 
ent income, high school GPA, AP credits earned, ACT Composite, 
first-term GPA in college, first-to-second-year retention, four-year 
graduation, and six-year graduation. Table 1 shows statistically sig- 
nificant differences in the demographic characteristics of students 
who were in versus not in the honors college (all p’s < .001). Hon- 
ors students were disproportionately more likely than non-honors 
students to be women. African American and Latino/a students 
were disproportionately not in the honors college, whereas Asian 
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American students were disproportionately in the honors college; 
White students were equally likely to be in the honors college as 
not. There were proportionally fewer first-generation college stu- 
dents among honors students than non-honors students; however, 
the average family income was lower for honors than non-honors 
students. In terms of academic preparedness for college, not sur- 
prisingly, honors students were much better prepared as reflected 
in significantly higher high school grades, more AP credits earned, 
and higher ACT scores (all p’s < .001). When examining honors col- 
lege membership, logistic regression revealed that all these factors 
were significant in the statistical model. Specifically, honors college 
membership was predicted by being a woman, being slightly older, 
not being an underrepresented minority student, having a lower 
family income, not being a first-generation college student, having 
a higher high school GPA, earning more AP credits, having a higher 
ACT composite score, and having a higher writing placement, all 
significant ps < .001. Finally, in terms of outcome variables, Table 
1 shows that honors college students also had far greater academic 
success than non-honors students. As expected, they earned higher 
grades in their first term; earned more credits in their first year; 
and had higher first-to-second-year retention, four-year gradua- 
tion (notably 69% versus 24%), and six-year graduation rates (85% 
versus 53%), 


Effects of Honors College Participation Starting in the 
First Semester of College on Measures of Success 


First-Term GPA 


Our main model-testing analyses revealed that, as predicted, 
even after statistically controlling for student background char- 
acteristics, participation in the honors college that started during 
the student’s first semester was positively associated with greater 
student success in terms of first-term GPA, first-to-second-year 
retention, credits earned in the first year, and graduation within 
four years and six years. (Final steps of models are presented in 
Table 2.) Specifically, we used hierarchical multiple regression to 
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examine the associations for first-term GPA. Age and gender were 
entered in the first step of the model, and gender was a significant 
predictor (R’ = .004, F(2, 15106) = 27.19, p< .001), such that women 
earned a higher GPA than men. In the second step, race/ethnicity 
variables were added to the model and were significant predictors 
(AR? = .037, F(3, 15103) = 129.15, p < .001), with Latino/a and Afri- 
can American students earning lower first-term grades than other 
students. In the third step, parent income and first generation in 


TABLE 1. MEANS AND PERCENTAGES FOR ALL STUDY VARIABLES 
Honors  Non-Honors 

Demographics College College 1G 
Gender 

Male 38% 46% 48,94*** 

Female 62% 54% 
Age in years 18.0 18.1 G27 
Race/ethnicity* 

White 37% 37% 0.19 

African American 5% 10% 16320" 

Asian American 39% 23% DIL SO 

Latino/a 11% 23% 184.28*** 
First-generation students 21% 40% 289:81"=* 
Parent income $64,461 $85,836 15.84°** 
High school GPA 3.52 gall -47,29%** 
AP credits earned 0.76 0.18 -65.12*** 
ACT composite 28.75 P32 -47, 940" 
Writing placement 4.75 3.99 -62.09"** 
First-term GPA By, 2.56 -50.32*"* 
First-year credits earned 30.06 22.45 -24,59*** 
First-to-second-year retention 96% 77% 475,65*** 
Four-year graduation 69% 24% — 1,292.61*** 
Six-year graduation 85% 53% 266.81*** 


*9 $05, *p <.01, ***p< 001 
“Race/ethnicity was dummy coded so that Asian American was the comparison group. 
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Tasle 2. FINAL STEP OF HIERARCHICAL AND Locistic REGRESSION FOR VARIABLES 
Honors CoLtece PARTICIPATION IN First TERM 


First-Term GPA? First-Year Credits Earned 


Predictors N03) BB SE(B) 
(Constant) 0.24 
Age -0.01 0.02 -0.01 -0.32 0.15 -0.02* 
Gender (Male = 0) 0.08 0.01 0.04*** 0.44 0.15 (Ope 
Race/ethnicity * 


African American SU 20s eeU0SsmeesU UG: Nee JySm 200 nema O.205 8 

White 0.10 0.02 0.05*** -0.76 0.18 -0.04"** 

Latino/a -0.09 0.02 -0.04"* -298 021 -0.13*"* 
Parent income 0.00 0.00 0.03*** 0.00 ~—-0.00 0.01 
First generation in college = 00 002 00s 0:83) 0G 0.045" 
High school GPA 0.68 0.02 O29 tors Oe Uae 
AP credits earned 0.27 0.02 Os 95 08 Cit 
ACT composite 0.01 0.00 0.04e** 0.16 0,03 0.06*** 
Writing placement 0.05 0,01 0,03*** as (i 0.02* 
Honors college first term 0.19 0.04 0.06*** 168 0.38 0057 
Honors x African American = 0.37.11 0103""* 449 1.08 003°" 
Honors x White 0.00 0.05 0.00 Ozu U2 0.01 


0.14 0.027 
0.21 


15,109 


010343* 
0.16 
15,109 


Honors x Latino/a 


*p $05, **p <.01, ***ps 001, tp < .06 

*Race/ethnicity was dummy coded so that Asian American was the comparison group. 

> The analysis included different cohorts of students for each outcome variable: First-Term GPA (started between 
(started 2006-2012, inclusive), 4-Year Graduation (started 2006-2009, inclusive), and 6-Year Graduation (started 
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PREDICTING COLLEGE Success UsiNG Five DIFFERENT OUTCOME VARIABLES AND 


Ist- to 2nd-Year Retention 4-Year Graduation 6-Year Graduation 


Odds Ratio ORE Odds Ratio 
| es) 31033) (a) B SE(B) (Ga) BSE(B) (a) 
0.85 0.06** -1.32 0.27 
0.00 0.05 1.00 -0.15 0.06 0.86** -0.07 0.07 0.93 
-0.05 0.04 0.95 0.34 0.05 Ngee -0.09 0.07 0.91 


-0.53 0.08 0.59%** -048 0.11 0.62*** -0.46 0.13 0.63*** 


-0.43 0.06 Ose 0.13 0.07 1.14 -0.08 0.09 0.92 

-0.49 0.07 0.61** = -0.48 0.09 0.62* -0.22 0.11 0.80* 
0.00 0.00 1.00 0.00 0.00 LO 0.00 0.00 1.00 

-0.10 0.05 0.91* -0.10 0.06 0.90 -0.10 0,07 0.90 


0.73 0.05 Osea 0.95 0.07 Aspe 0.91 0.09 2.48*** 
0.68 0.06 Lg 0.80 0.06 Deda 0.69 0.09 Loree 


0.00 0.01 1.00 0.01 0.01 1.01 0.00 0.01 1.00 
-0.07 0.04 0.94 0.17 0.05 IL GPS 0.02 0.06 1.02 

0.62 0.19 Liye 0.79 0.14 Ones 0.68 0.27 1.98% 

0.78 — 0.55 2.18 0.37 0.40 1.45 -0.44 0.65 0.64 
-0.28 0.24 0.76 -0.11 0.20 0.89 -0.71 0.34 0.49* 


0.80 
0.09 
15,109 


1.02 =a 0.81 


2006 and 2012, inclusive), First-Year Credits Earned (started between 2006 and 2012), Ist- to 2nd-Year Retention 
2006-2007, inclusive). 
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college were added and were significant (AR* = .009, F(2, 15101) 
= 112.68, p < .001), such that students with higher parent income 
and who were not first generation earned higher grades than oth- 
ers. In the fourth step, high school background characteristics 
(high school GPA, number of AP credits earned, ACT Composite 
scores, writing placement) were positively associated with first- 
term grades (AR’ = .156, F(4, 15097) = 354.37, p < .001). In the 
fifth step, honors college participation was added and had a sta- 
tistically significant effect (AR? = .004, F(1, 15096) = 331.93, p < 
.001)—even after accounting for the variance associated with the 
other variables—with honors participation being associated with 
higher grades. In the sixth step, the interaction terms between 
race/ethnicity and honors college participation were added, and 
the interaction term for African American was significant (6 = .03; 
AR’ = .001, F(1, 15093) = 266.74, p < .001, total R’ = .21), indicat- 
ing that the statistical effect of honors involvement was larger for 
African American students compared to Asian American students. 
The positive effect of honors college involvement in the first term 
was larger for African American students when compared to Asian 
American students. African American students in the honors col- 
lege their first semester saw a 0.37 increase in first-semester GPA 
compared to African American students not in the honors college. 


First-Year Credits Earned 


We used the same analytic approach (with the same variables 
entered in the same steps) for the different dependent measure 
of credits earned during the first year. A similar pattern of find- 
ings emerged (see Table 2 for the final step of hierarchical multiple 
regression analysis) with two exceptions: (1) parent income was not 
significantly associated with first-year credits earned, and (2) in the 
final step, both the interaction between honors participation and 
Latino/a student ethnicity and the interaction of honors participa- 
tion and African American race were statistically significant (F (1, 
15093) = 190.95, p < .001, total R’ = .16). That is, the positive effect 
of honors college involvement was significantly larger for Latino/a 
students as well as for African American students compared to 
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Asian American students. African American students in the hon- 
ors college their first semester, on average, earned 4.49 more credits 
in the first year compared to African American students who were 
not in the honors college, while Latino/a students earned 2.52 more 
credits in their first year when compared to Latino/a students not 
in the honors college. 


Retention and Graduation 


We used logistic regression to examine the association of honors 
college participation on first-to-second-year retention, four-year 
graduation, and six-year graduation (Table 2). For all three, honors 
involvement was statistically significant: students who participated 
in the honors college had a higher likelihood (almost two times 
more likely) of persisting to the sophomore year (odds ratio [OR] 
= 1.87), and of graduating at four (OR = 2.20) and six years (OR = 
1.98), even after accounting for the other factors included in the 
model. In addition, high school GPA and AP credits earned, as well 
as some race/ethnicity variables, were also significantly associated 
with retention and four- and six-year graduation. The interaction 
terms between racial/ethnic groups and honors college partici- 
pation were largely not significant, as was ACT composite score, 
indicating that these were not statistically associated with retention 
and graduation. Analyses testing the model predicting four-year 
graduation revealed that some additional variables were signifi- 
cantly related, including age (younger students were more likely to 
graduate within four years than older students), gender (women 
were more likely to graduate within four years than men), parent 
income (students with higher parent income were more likely to 
graduate within four years than students with lower parent income), 
and writing placement (those with higher scores were more likely 
to graduate within four years than those with lower scores). But 
these variables were not significantly associated with retention nor 
six-year graduation. 
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Effects of Proportion of Time Spent in the Honors 
College on Measures of Success 


The next series of logistic regression analyses tested a similar 
model that had the same steps as above, but this series used as the 
main predictor variable the proportion of time that each student 
spent in the honors college (instead of the independent variable 
of whether students had entered the honors college in their first 
year). For both four- and six-year graduation, increased time in the 
honors college was associated with greater likelihood of graduation 
(Table 3). Students who were in the honors college for a greater 
proportion of their time in college were four times more likely to 
graduate within four years (OR = 4.10) and almost three times 
more likely to graduate within six years (OR = 2.83). No interac- 
tion terms were significant in either analysis. Control variables in 
this model followed the same pattern as reported above for analyses 
using honors college participation defined as college membership 
starting in the first term of college. 


Effects of Honors College Participation at Any Point on 
Measures of Success 


In the next analyses, we used logistic regression to determine if 
honors college membership at any time during a student's college 
tenure affected four- and six-year graduation (Table 4). Honors stu- 
dents were significantly more likely—three times more likely—to 
graduate within four years than non-honors students (OR = 3.10). 
No interaction terms were statistically significant. Other control 
variables in this model followed the same pattern for four-year 
graduation as reported above when the honors college variable was 
defined as participation starting in the first term of college. 

A similar logistic regression also revealed that students who 
were in the honors college at any time during college were three 
times more likely to graduate within six years than those who were 
not in the honors college (OR = 3.29). No interaction terms were 
statistically significant. Again, the other variables in this model fol- 
lowed the same pattern for six-year graduation as when the honors 
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college variable was defined as participation starting in the first 
semester. 


DISCUSSION 


Without question honors college students are more success- 
ful than non-honors students. Some critics have argued that the 
enhanced success of honors students is not due to honors educa- 
tion per se, but instead due to the preexisting characteristics of 
honors students themselves: they are better prepared and socio- 
economically advantaged, they have higher entering standardized 
test scores, and they are more likely to be white or Asian. On the 
contrary, our analyses show that such an explanation, which leaves 
little justification for supporting honors colleges and programs 
on university campuses, is not accurate. Indeed, this study shows 
that honors education has a statistically significant positive effect 
on student success above and beyond all other background char- 
acteristics studied, including prior academic preparation (e.g., as 
reflected in high school grades, writing class placement, and ACT 
scores) and student and parent demographics (e.g., first generation 
in college). This was true for success defined five different ways: 
grades earned in the first semester, credits earned in the first year, 
first-to-second-year retention, 4-year graduation, and 6-year grad- 
uation. Furthermore, and of great importance in a nation where a 
significant gap in the success of underrepresented students versus 
others exists, we found that the positive effects of honors college 
membership were more pronounced for African American and 
Latino/a students for some indicators of success. 

Although our goal was not to identify the specific components 
of honors programs that increase academic success, theoretically, 
the explanation might lie in the centerpiece of honors education: 
the many academically and socioemotionally supportive prac- 
tices. These include high-impact practices such as small interactive 
classes, first-year seminars, service activity requirements, and cap- 
stone research requirements; all of these practices help students 
engage more with college, their peers, and their professors (Inkelas 
and Weisman 2003; Freeman et al. 2007; Kuh 2008; Mayhew et al. 
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2016). For example, this particular university honors college has 
a mandatory honors freshman seminar, which focuses on campus 
resources and engagement, and required first-year core seminars, 
both of which help build relationships between students and fac- 
ulty and advisors. In turn, these enhanced relationships might help 
students feel an increased sense of belonging—an important com- 
ponent of academic mindsets, which is strongly associated with 
academic achievement in college and persistence to the second year 
(Han, Farruggia, and Moss 2017; Walton and Cohen 2011). Other 
honors experiences (e.g., community projects, student organiza- 
tions, and leadership) also promote more engagement and probably 
more perceived support and belonging. 

One could argue that there are additional individual differences 
between honors and non-honors students that we did not account 
for. For example, Seifert et al. (2007) found that students who par- 
ticipate in an honors program had increased critical thinking skills, 
skills in mathematics, and composite cognitive development, and 
Scager et al. (2012) found that honors students had more desire to 
learn, drive to excel, and creativity compared to non-honors peers. 
Perhaps honors college students are more inherently motivated, 
both to apply to and gain admission to college and to study and be 
successful once there. While no study can control for everything, 
future studies should certainly include such variables. Even so, 
there are several good reasons for not expecting such potential dif- 
ferences to explain our effects. For example, students’ high school 
grades were statistically controlled, and grades reflect a strong 
motivational component. Of even more importance, although hon- 
ors college participation at any point in the students’ college careers 
led to a higher chance of graduating in four or six years, the more 
time students spent in this honors college, the more successful they 
were in terms of the likelihood of graduating. This would not be 
true if the honors college programs and resources, which included 
required honors activities each semester, were not at least partially 
responsible for the increased success. Thus, we have confidence that 
the background characteristics we included are reasonable proxies 
for a host of factors, such as those considered in holistic admissions 
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processes, that, when controlled, help us to better understand 
the unique association between honors participation and college 
success. 

It is interesting that the impact of honors college participation 
is stronger for indicators of persistence in college (retention and 
graduation—arguably the most important variables we studied) 
rather than academic performance as measured by first-term GPA 
and first-year credits earned. (Statistically, this is indicated by the 
relatively small, but statistically significant and consistent, B’s and 
small amounts of variance explained for performance, but larger 
Exp (’s for retention.) The types of support provided by the honors 
college may help students manage barriers to college graduation 
more so than the barriers to academic performance in college. This 
interpretation is logical, given that honors college students are gen- 
erally highly academically prepared, but they will still face other 
challenges that all students face, such as economic barriers and 
developing social relationships. 

The statistically significant interactions between racial/eth- 
nic background and honors college membership are particularly 
interesting and important. Honors involvement was beneficial for 
all students, but it was especially important for African American 
students in terms of first-term GPA, and for African American and 
Latino/a students in terms of number of first-year credits earned. 
Seifert et al. (2007) found a similar effect regarding first-year 
outcomes, but failed to examine long-term outcomes, such as grad- 
uation, as we did. As previous research has indicated, regardless of 
honors involvement, racial and ethnic minority students, compared 
to White students, tend to engage less with faculty (Anaya and Cole 
2001) and to have less access to resources including money for 
tuition (Bastedo and Gumport 2003). Honors programs provide 
such resources—more opportunities for faculty engagement, more 
resources that help academically, and often more scholarships—all 
of which are needed more by underrepresented students as a group 
than by other students. Thus, underrepresented students benefit 
even more than others from an honors college. Given the significant 
achievement gap between underrepresented and majority students 
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in this country, it would have been encouraging in this study sim- 
ply to see equivalence in the effects—our finding of greater impact 
for African American and Latino/a students is truly important in a 
meaningful, practical sense. 


CONCLUSIONS AND FUTURE DIRECTIONS 


This research documents the positive association between hon- 
ors education and student success over and above other factors, and 
it demonstrates that honors education is even more beneficial for 
underrepresented minority students than for some other students. 
Honors programs are campus models for undergraduate success 
programming, not simply unneeded extra resources for students 
who already have a competitive advantage. 

Future research could expand the definition of student success 
to include elements such as lifelong learning, later-life civic engage- 
ment, graduate and professional school matriculation and success, 
or career development, and it could begin to tease apart the various 
features of the honors experience that contribute most to student 
success, with qualitative and quantitative methods. Future research 
should also continue to identify factors that explain student success 
of both honors and non-honors students. We have identified one 
important piece of the complex, multiply determined puzzle, but 
more research is needed to expand the growing evidence converg- 
ing on a complete answer to the question of what makes students 
successful. 
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lexander Astin’s Inputs-Environment-Outcomes (I-E-O) model 

for longitudinal study of student success in higher education 
challenges researchers to account explicitly for the wide range of edu- 
cational, social, and cultural backgrounds that students bring with 
them to college. Astins approach factors in an understanding that 
educational outcomes are associated not only with the various edu- 
cational environments to which students are exposed during their 
college years, but also with the inputs of these students—the factors 
that shaped them long before they first arrived in a university class- 
room. Meaningful conclusions concerning factors that contribute to 
student success must take into account the complex interactions of 
all three of the I-E-O components. Inputs precede and inform stu- 
dent choices of and attitudes toward their environments, and both 
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play significant and interrelated roles in shaping educational out- 
comes for each student (Astin 1993). 

Applied to questions of the impact and value of honors edu- 
cation, the I-E-O approach demonstrates the need to be expansive 
and iterative, rather than reductive, in designing strategies to assess 
the impact of honors educational practices. Honors administra- 
tors, staff, and faculty often promote and defend the value of their 
programs by appealing to outcomes of honors students, such as 
retention and graduation rates, that are far superior to those among 
the general student population. Attributing those superior outcomes 
to the supposed benefits of honors educational practices, however, 
rings hollow when the differential inputs between honors student 
populations and non-honors student populations are not taken 
into account. Success in the classroom typically made the honors 
students eligible for honors education in the first place, and those 
students would be expected to persist and graduate at much higher 
rates than the general student population, with or without honors 
educational experiences. In order to measure the impact of hon- 
ors educational practices on student success outcomes, researchers 
must control for these inputs. One way to accomplish this task is to 
use a comparison group that resembles the honors student group in 
terms of academic preparation and readiness for college. 

To date, only a handful of studies have controlled for student 
inputs in this way by comparing honors students to high-achieving, 
non-honors subgroups among general student populations, that is, 
students with prior educational attainment levels that are similar 
to those of the honors students in the study (e.g., Shusok 2006). 
Keller and Lacy (2013) compiled data concerning students enter- 
ing the honors program at Colorado State University (CSU) from 
2005 to 2008, comparing them to a similarly sized control group of 
high-achieving incoming CSU students who did not participate in 
honors but who, as a group, had average test scores and high school 
GPAs comparable to the honors student cohort. They found only 
a slight difference in second-year retention between the honors 
(92.9%) and high-achieving non-honors (87.9%) groups. Of greater 
interest to Keller and Lacy (2013) was a more dramatic advantage 
for the honors cohort over the comparable non-honors population 
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in terms of four-year (64.2% vs. 55.8%), five-year (81.9% vs. 69.6%), 
and six-year (88.9% vs. 74.9%) graduation rates from CSU. 

By contrast, a similar comparative study by Slavin, Coladarci, 
and Pratt (2008) involving honors and high-achieving non-hon- 
ors cohorts at the University of Maine found only an insignificant 
advantage for the honors students in terms of four-year graduation 
rates (64% vs. 60%), but a genuine and significant honors advantage 
in second-year retention (94% vs. 85%). More troubling were the 
findings of Cosgrove (2004), who studied a group of 112 honors 
students and 108 comparable non-honors students at three differ- 
ent regional universities in the Pennsylvania State University system 
from 1997 to 2002. Of the 112 honors students, only 30 graduated 
with all honors requirements completed within five years. Among 
the partial honors completers (i.e., those who started in honors but 
dropped honors at some point in college) Cosgrove (2004) found 
five-year graduation rates (82%) to be only slightly higher than 
those of the group of similar non-honors students (76%). Data of 
this sort collected thus far are inconclusive in terms of being able to 
assert with confidence the value added of honors educational expe- 
riences, much less which features of honors environments most 
closely correlate with student success outcomes. 

This study constitutes a fresh empirical contribution to this 
conversation, grounded in an extensive database of honors and 
non-honors students. The honors group consists of 590 first-year 
students entering the Eastern Kentucky University (EKU) Honors 
Program in their first semester between fall 2010 and fall 2015, 
while the non-honors group contains 610 first-year students enter- 
ing EKU during that same period with a prior educational profile 
that resembles that of the honors students (in ways specified later in 
the “Study Design” section) but who did not participate in the hon- 
ors program. Differences in second-year retention and graduation 
rates were more dramatic in favor of the honors group than those 
found by Keller and Lacy (2013) and Slavin et al. (2008). Follow- 
ing examination of these data, this study also takes a preliminary 
step toward illuminating more clearly the environments of the EKU 
Honors Program and the specific effects of its high-impact educa- 
tional practices and programming (Kuh 2008). Students who chose 
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to be involved in at least one of the additional high-impact prac- 
tices of the honors experience at EKU are distinguished from those 
who did not participate in one of these activities and are therefore 
labeled as less involved. Within this distinction, strong associations 
are found between participation in these high-impact practices and 
student success outcomes. 


EASTERN KENTUCKY UNIVERSITY AND THE EKU 
HONORS PROGRAM 


Located in Richmond, Kentucky, on the southern edge of the 
Lexington metropolitan area, Eastern Kentucky University is a pub- 
lic comprehensive “master’s” university with a total enrollment of 
just under 17,000 students, including approximately 14,200 under- 
graduates. Growing from its normal school or teacher college roots, 
EKU has traditionally drawn heavily from its service region, the coal 
towns of Appalachian southeastern Kentucky. With the declining 
populations, however, of those areas in recent decades, the university 
has increasingly marketed itself in the region's larger cities. Today, 
only about one third of EKU students come from EKU’s traditional 
service region. Roughly half of EKU students in the 2016-2017 aca- 
demic year came from the nearby metropolitan areas of Louisville, 
Cincinnati, and Lexington. In addition, nearly one third of all EKU 
students identify as first-generation college attendees. 

The EKU Honors Program, founded in 1988, enrolls approxi- 
mately 500 students whose social and demographic profile generally 
matches that of the student body as a whole. The average composite 
ACT score of incoming honors program students is 28-29, while the 
average unweighted high school GPA is 3.8-3.9. The EKU Honors 
Program provides an excellent case for examining the value added 
to the undergraduate experience via high-impact honors pedagogies 
and programming for two reasons. First, the EKU Honors Program 
provides a uniquely intensive high-impact curriculum in which 
honors seminars are team-taught by faculty from two different aca- 
demic disciplines. Every student going through the full program 
takes 18 credit hours of the total 25 hours of honors curriculum 
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within the context of these team-taught interdisciplinary honors 
seminars. In our most recent external program review, the outside 
evaluators, both past presidents of the National Collegiate Honors 
Council (NCHC), said the following about our honors curriculum 
in the “EKU Honors Program External Program Review” (2015): 


The honors curriculum at EKU is a distinctive and powerful 
model of exemplary honors education. With the empha- 
sis on team-taught interdisciplinary courses, it exemplifies 
characteristics valued nationally in honors pedagogy. Most 
honors programs and colleges have one or two interdisci- 
plinary courses required in the curriculum; at EKU Honors, 
interdisciplinarity and team-teaching are true hallmarks, 
and the program is well respected nationally. 


Second, the EKU Honors Program has an unusually rich tra- 
dition of providing opportunities for undergraduate research 
presentations at venues such as the annual meetings of the NCHC, 
the Southern Regional Honors Council (SRHC), and the National 
Council of Undergraduate Research (NCUR). Since 1990, more 
than 1,000 EKU Honors Program students have made presentations 
at the annual meeting of the NCHC, making the program the leader 
in NCHC student presentations among all honors programs and 
colleges nationwide. The program has a $1.8 million endowment 
dedicated specifically to creating travel and learning opportunities 
for honors students. Income from this endowment each year is spent 
on national and regional conference presentation travel, as well as 
study abroad and study away grants for which honors students may 
apply. In short, all students in the EKU Honors Program experi- 
ence a distinctive high-impact educational experience via their 18 
credit hours of interdisciplinary, team-taught coursework, allowing 
for a clear distinction to be made between them as a group on the 
one hand and the comparable non-honors group on the other. Fur- 
thermore, within the honors student group itself, the exceptionally 
high numbers of students who participate in additional high-impact 
experiences of their choosing, such as undergraduate research con- 
ference presentations, allow for a meaningful distinction to be 
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drawn between honors students who do and do not choose such 
activities. Controlling for these inputs, the data presented here allow 
for meaningful insight into the effects of high-impact honors peda- 
gogical and programming practices on student success outcomes. 


STUDY DESIGN 


In an attempt to determine the value added of the EKU Honors 
Program experience, two groups were examined. The first consists 
of students who began in the EKU Honors Program in their first 
semester between fall 2010 and fall 2015. Honors program records 
were used to compile this data set, which comprises 590 students 
who started in the honors program at the beginning of their college 
career. For each of these students, we consider six outcome mea- 
sures: (1) second-year retention within the honors program, (2) 
second-year retention at the university, (3) graduation as an honors 
scholar from the EKU Honors Program, (4) graduation from the 
university with a bachelor’s degree within four years, (5) graduation 
from the university with a bachelor’s degree within five years, and 
(6) involvement within the honors program. 

We measure second-year retention within the honors program 
and at the university as being a member of the honors program 
and/or enrolled at the university in the fall of a student’s second 
year after matriculation. Honors program records were used both 
to determine second-year retention within the honors program and 
honors scholar graduation, meaning a student's having completed 
the honors curriculum, successfully written and presented an hon- 
ors thesis, and graduated from the university. We used university 
records to determine second-year retention within the university, 
as well as four- and five-year graduation rates from EKU. Gradua- 
tion rate data are limited by the fact that five-year graduation data 
are only readily available for the fall 2010-fall 2012 cohorts, and 
four-year graduation data are only available for the fall 2010-fall 
2013 cohorts, due to the timing of this study, with data collected 
during the summer of 2017. 

To be considered highly involved in the honors program, stu- 
dents participated in one of two groups of high-impact activities. 
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The first consists of three conferences at which students could pres- 
ent research during their time at EKU: the annual meeting of the 
SRHC, the annual meeting of the NCHC, or the annual meeting 
of NCUR. As previously discussed, the rich tradition of student 
presentations at these conferences made this accomplishment a 
natural marker of involvement within the EKU Honors Program. 
The second group of activities includes opportunities for stu- 
dent leadership within the honors program, namely three specific 
endeavors. Students who served as officers in the Honors Student 
Advisory Council (HSAC), the student governing body of the hon- 
ors program, were considered highly involved. Between five and 
eight students each year fill a variety of offices on the HSAC. These 
students are elected by their peers each year and plan and execute 
service and social activities for the honors program. Peer mentors 
for the Honors Seminar (HON 100), the first-year student suc- 
cess seminar for honors students, were also included in this group. 
This cohort would typically include five to six students each year. 
These students are selected by the instructor of the section they 
mentor and perform a variety of activities, including meeting with 
first-year students and serving as sources of valuable honors infor- 
mation from the student perspective. Students selected as Honors 
Ambassadors made up the last part of the highly involved group. 
Serving as a resource in recruiting prospective honors students, ten 
students are selected each year, and they travel to events with the 
program coordinator and university admissions staff. Records of 
participation in all of these activities were consulted to create this 
group of 113 highly involved students within the EKU Honors Pro- 
gram, 19.5 percent of the total group of 590 honors students. One 
limitation should be noted when discussing this measurement of 
involvement, and that is that the fall 2014 and fall 2015 cohorts of 
students still have opportunities to participate in these activities. 
This total number of highly involved students may increase if these 
data are analyzed again in a few years. 

Our study design involves identification of a second group: 
comparable non-honors students. The goal behind establishing the 
second group was to identify a sample of EKU students who did not 
participate in the honors program but who came into the university 
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similarly academically prepared in terms of widely recognized mea- 
sures of college preparedness. This data set allows for comparisons 
of students who should have comparable inputs using Astin’s I-E-O 
model. Creation of this group involved three subsets of students. 
The first consists of 12 students who applied to and were accepted 
to the EKU Honors Program from 2010 to 2015, matriculated to 
EKU, but chose not to participate in the honors program. This 
cohort is likely the closest one can get to a true control group: these 
students met the criteria for becoming an honors program student, 
were selected to do so, but never entered the program. Because 
this number is small, the Office of Institutional Research at EKU 
provided the other two subgroups of students for this non-honors 
group. One consists of 299 students who enrolled in the university 
First-Year Writing Seminar (English 105) during 2010-2015. Eng- 
lish 105 is an accelerated writing course with a prerequisite of an 
ACT English subscore of 28 or higher or an SAT verbal score of 660 
or higher. Students who earn an A or B in English 105 receive six 
credit hours and fulfill their written communication general educa- 
tion requirements with one course rather than taking both English 
101 and English 102 (EKU Undergraduate Catalog 2017). English 
105 is often presented as an alternative to the standard first-year 
writing course (English 101) for academically well-prepared first- 
year students during their initial orientation to the university, and 
because of its test score prerequisites, it seems like a natural choice 
to include in the non-honors group. Since students choose to enroll 
in English 105 (rather than English 101), this group provided 
students who seemed to be seeking more in-depth educational 
experiences. In turn, adding these students to the group mitigates 
to some extent a possible limitation of the research: that students 
must choose to apply to the honors program and may have higher 
levels of motivation to persist. 

To bring this group closer to the total of 590 honors students, 
institutional research staff provided an additional 299 students 
who are a random sample of all students entering EKU during fall 
2010-fall 2015 with a 28 or higher composite ACT score and a 
3.8 or higher unweighted high school GPA. While the EKU Hon- 
ors Program does not have minimum ACT or high school GPA 
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requirements, these numbers are roughly equal to the incoming 
class averages of honors program students. These three approaches 
generated a comparable non-honors group of 610 students. 

Institutional research provided outcome measures similar to 
those described above for each student in the non-honors data 
set, using the Banner student information system: (1) second-year 
retention at the university, (2) graduation from the university with 
a bachelor’s degree within four years, and (3) graduation from the 
university with a bachelor’s degree within five years. Again, gradu- 
ation rate data are limited by the timing of this study, so five-year 
graduation data are only considered for the fall 2010-2012 cohorts, 
and four-year graduation data are only considered for the fall 2010- 
fall 2013 cohorts. 

Once all data had been collected, several comparisons were 
made, and we present those in the results section below. We made 
comparisons between the honors and comparable non-honors 
groups, as well as between the highly involved honors and the less 
involved honors students. 


RESULTS 


Honors vs. Comparable Non-Honors 
Second-Year Retention 


We first turn to an analysis of a standard measure of second- 
year retention. This measure is the only one in which the full data 
set (fall 2010-fall 2015) of both groups could be considered. Results 
of this comparison are presented in Figure 1. Of 590 honors stu- 
dents, 565 (95.8%) returned to EKU for the start of their second 
year. Only 486 (79.7%) of the 610 comparable non-honors students 
returned to EKU, yielding a difference of 16.1 percentage points. 

Figure 2 presents a line graph of the second-year retention rates 
over time for honors versus comparable non-honors students for 
fall 2010-fall 2015 incoming first-year students. The graph high- 
lights the gap between honors and non-honors students over time 
while also showing that the rates for the two groups generally fol- 
low the same pattern. 
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Some possible approaches to comparing the honors and non- 
honors groups, such as z-tests for two proportions or two-sample 
confidence intervals, require independent random samples. The 
honors group in the study included every honors student for the 
given time period. Since population data are available for honors 
students, second-year retention rates for the fall 2010-fall 2015 
honors first-year classes are known; no uncertainty about these 
parameter values exists for this time frame. We calculated confi- 
dence intervals for second-year retention rates for each of the fall 
2010-fall 2015 non-honors first-year classes using the data provided 
by institutional research staff and compared them to the population 
proportions for the honors students. (This process is similar to con- 
ducting one-sample tests using the non-honors data for the sample 
and treating the honors proportions as the null values, but without 
the limitations of the tests detailed in the Limitations section.) We 


Ficure 1. EKU Seconp-YEAR RETENTION FOR Honors vs. COMPARABLE 
Non-Honors STUDENTS 


F10-15 F10-15 
Honors Students | Comparable Non-Honors Students 
(n = 590) (n =610) 
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used a confidence level of 99.17 percent for each interval based on a 
Bonferroni correction (1 - .05 + 6 = .9917). The Bonferroni correc- 
tion accounts for the fact that multiple comparisons have been made 
(Weisstein n.d.). Only one of the six confidence intervals contains 
the corresponding honors retention rate, and that one (fall 2013) 
barely does, suggesting that non-honors retention rates differ from 
the honors retention rates for most, if not all, of the years studied. 
(Note that the Bonferroni correction produces conservatively wide 
confidence intervals. We summarize those results in Table 1.) 


Honors vs. Comparable Non-Honors Graduation 


We compared four- and five-year graduation rates between the 
honors and non-honors groups for the fall 2010-fall 2012 cohorts, 


Ficure 2. EIU Seconp-YEAR RETENTION RATES OVER TIME FOR Honors 
vs. COMPARABLE Non-Honors STUDENTS FOR 
Fatt 2010-2015 INcomina First- YEAR STUDENTS 


Fall 2010 Fall2011 Fall2012 Fall2013 Fall2014 Fall 2015 
Cohort 


== Honors === Comparable Non-Honors 


103 


PATTON, COLEMAN, AND Kay 


based on the availability of graduation data as previously discussed. 
We have presented the results of that comparison in Figure 3. Com- 
pared to the second-year retention data, more significant gaps are 
evident when comparing graduation rates. After four years, 72 per- 
cent of honors students (185 of 257) had earned an undergraduate 
degree from EKU. Only 46.9 percent of the comparable non-honors 
students (172 of 367) had graduated during that same time period, 
a difference of 25.1 percentage points. After five years, that gap had 
widened by almost ten percentage points. The honors group had 
a five-year graduation rate of 87.2 percent (224 of 257), while the 
non-honors group graduated 52.3 percent (192 of 367) during the 
same time period. 

We also examined the four-year graduation rate for the cohorts 
beginning in fall 2010-fall 2013, and we present those results in Fig- 
ure 4. Due to the time frame of data collection, we could examine 
only four-year graduation rates for the cohorts entering between 
fall 2010 and fall 2013; students in these cohorts had not had the 
full five years to graduate at the time of our data collection during 
the summer of 2017. After four years, 73.7 percent of honors stu- 
dents (260 of 353) had earned an undergraduate degree from EKU, 
compared to 45.5 percent of non-honors students (200 of 440). That 
represents a difference of 28.2 percentage points, approximately the 


Tasle 1. Comparison oF EKU Honors RETENTION RATES WITH 
CONFIDENCE INTERVALS FOR COMPARABLE Non-Honors 
RETENTION RATES FOR FALL 2010—Fatt 2015 


Second-Year Comparable Comparable 99.17% 
First-Year Retention Rate | Non-Honors Non-Honors iyicitsites 
Class for Honors Sample Size  RetentionCount —_Interval* 


Fall 2010 100.0% 71.7%, 90.2%) 
Fall 2011 93.4% 68.6%, 87.3%) 


Fall 2013 95.8% 73.9%, 96.0%) 
Fall 2014 98.4% 71.5%, 94.3%) 
Fall 2015 90.4% (67.6%, 89.9%) 


*C] for the second-year retention rate for comparable EKU non-honors students (overall a = .05). The 
confidence level of 99.17% is based on a Bonferroni correction (1 - .05 = 6 = .9917). 
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same size as that witnessed for the four-year graduation rate in the 
fall 2010-fall 2012 group. 

Again, since population data are available for honors students, 
five-year graduation rates for the fall 2010-fall 2012 honors first- 
year classes are known. Confidence intervals for five-year graduation 
rates for each of the fall 2010-fall 2012 non-honors first-year classes 
were computed using the data provided by institutional research 
staff. We used a confidence level of 98.33 percent for each inter- 
val based on a Bonferroni correction (1 - .05 + 3 = .9833). None 
of the three confidence intervals contain the corresponding hon- 
ors five-year graduation rate, suggesting that non-honors five-year 
graduation rates differ from the honors five-year graduation rates 
for the years in question. The results are summarized in Table 2. 


Ficure 3. EKU Four-Year AND Five-YEAR GRADUATION RATES FOR 
Honors vs. CompaRABLE Non-Honors STUDENTS FOR 
Fatt 2010—Fatt 2012 INcomiNG First-YEAR STUDENTS 


4-Year 5-Year 
EKU Graduation Rate EKU Graduation Rate 


Ml F10-12 Honors Students (n = 257) 
F10-12 Comparable Non-Honors Students (n = 367) 
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Ficure 4. EIU Four-Year GRADUATION RATES FoR Honors vs. 
ComparABLe Non-Honors STUDENTS FOR 
Fatt 2010—Fatt 2013 INcomiNG First-YEAR STUDENTS 


F10-13 F10-13 
Honors Students Comparable Non-Honors Students 
(n = 353) (n = 440) 


TasLe 2. Comparison of EKU Honors Five-Year GRADUATION RATES 
WITH CONFIDENCE INTERVALS FOR COMPARABLE Non-Honors 
Five-YEAR GRADUATION RATES FOR FALL 2010—Fatt 2012 


Five-Year Non-Honors 98.33% 
First-Year Graduation Rate for Non-Honors Five-Year Oiyititates 
OER Honors Students © SampleSize GraduationCount —_Interval* 


48.2%, 69.2%) 


Fall 2010 92.5% ( 
Fall 2011 80.0% (39.7%, 60.3%) 


Fall 2012 90.7% (36.0%, 59.3%) 


*C] for the five-year graduation rate for high-achieving EKU non-honors students (overall a = .05). 
The confidence level of 98.33% is based on a Bonferroni correction (1 - .05 + 3 = .9833), 
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Four-year graduation rates for the fall 2010-fall 2013 honors 
first-year classes are known. Confidence intervals for four-year 
graduation rates for each of the fall 2010-fall 2013 non-honors 
first-year classes were computed based on the data provided by 
institutional research staff. We used a confidence level of 98.75 per- 
cent for each interval based on a Bonferroni correction (1 - .05 
+ 4 = .9875). None of the four confidence intervals contain the 
corresponding honors four-year graduation rate, suggesting that 
non-honors four-year graduation rates differ from the honors four- 
year graduation rates for the years included here. The results are 
summarized in Table 3. 


Honors Students: 
Highly Involved vs. Less Involved 


When comparing highly involved honors students to less 
involved honors students, we used only data from fall 2010 to fall 
2013, based on the previously discussed limitation that students in 
the fall 2014 and fall 2015 cohorts may still participate in the activi- 
ties used to measure involvement. Of 353 total students within 
these four groups, 113 students make up the highly involved honors 
student group. We compared the highly involved and less involved 
honors students on the following measures: second-year retention 
in the honors program, second-year retention at EKU, graduating 


TasLe 3. COMPARISON oF EKU Honors Four-YEAR GRADUATION RATES 
WITH CONFIDENCE INTERVALS FOR COMPARABLE Non-Honors 
Four-YEAR GRADUATION Rates FoR FALL 2010—Fatt 2013 


Four-Year Non-Honors 98.75% 


First-Year Graduation Rate for Non-Honors Four-Year Oiyicitates 
Class Honors Students  SampleSize GraduationCount Interval? 


Fall 2010 42.9%, 65.1%) 


( 
Fall 2011 (34.2%, 55.5%) 
( 


Fall 2012 29.0%, 52.9%) 
Fall 2013 (24.1%, 52.6%) 


*C] for four-year graduation rate for high-achieving EKU non-honors students (overall a = .05). The 
confidence level of 98.75% is based on a Bonferroni correction (1 - .05 + 4 = .9875). 
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as an honors scholar, and graduating from EKU within four years. 
We present these results in Figure 5. 

Participation in just one additional activity within the honors 
program appears to make a measured difference in most of these 
categories. The category with the smallest gap between highly 
involved honors students and less involved honors students is sec- 
ond-year retention at EKU, a gap of only 5 percentage points. It is 
worth noting, however, that 100 percent of highly involved honors 
students were retained at EKU at the start of their second year. This 
same cohort of highly involved honors students were also retained 
within the honors program for the second year at 100 percent, com- 
pared to 85.8 percent of less involved honors students (206 of 240). 

Wider gaps are observed in the four-year honors scholar gradu- 
ation result and the four-year graduation rate from EKU. Highly 
involved students graduated as EKU Honors Scholars within four 
years at a rate of 86.7 percent (92 of 113). That number drops 38.4 
percentage points for less involved honors students; they graduated 
as honors scholars at a rate of 48.3 percent (116 of 240). The gap in 
graduation rates narrows to 23.1 percentage points for the regular 
four-year graduation rate. Highly involved honors students earned 
their undergraduate degree in four years at a rate of 89.4 percent 
(101 of 113), while less involved students graduated in four years at 
a rate of 66.3 percent (159 of 240). 

In order to consider five-year graduation rates, we removed 
the fall 2013 cohort from the analysis and present those results in 
Figure 6. A total of 257 students make up the fall 2010-fall 2012 
cohorts, with 83 highly involved honors students and 174 less 
involved honors students. Again, highly involved honors students 
had a 100 percent second-year retention rate, both within the hon- 
ors program and at EKU, compared to less involved students at 
83.9 percent (146 of 174) and 95.4 percent (166 of 174), respec- 
tively. While the gap between highly involved and less involved 
honors students earning their undergraduate degree in five years 
is the smallest of the three graduation measures at 13.6 percentage 
points, we witness significant gaps once again in graduating from 
EKU in four years (23.6 percentage points) as well as graduating 
as an honors scholar (35.9 percentage points). (The honors scholar 
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graduation metric is for those graduating in five years total; thus 
no differentiation is made between those who graduated as honors 
scholars in four years versus five years.) 


DISCUSSION 


Analyzing these groups leads to some key points of discussion 
on the value added of participating in the EKU Honors Program, 
as well as involvement in some of the high-impact practices the 
program provides. Typically, honors program participants lead the 
overall university population in retention and graduation rates. 
This fact may be partially attributed to these students’ inputs, that 
is, being more academically prepared and having a mindset that 


Ficure 5. RETENTION AND GRADUATION RATES FOR HIGHLY INVOLVED 
vs. Less INVoLvVeD Honors STUDENTS FOR 
Fatt 2010—Fatt 2013 INcomiNG First-YEAR STUDENTS 


100.0 100.0 


95.0 
85.8 86.7 sus 
66.3 
48.3 


Honors First-Year EKU First-Year HonorsScholar EKU Four-Year 
to Second-Year toSecond-Year Graduation Graduation 
Retention Retention 


la Highly Involved (n = 113) Less Involved (n = 240) 
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predisposes them to academic success. The goal of this study was 
to explore whether a significant difference in these rates exists 
between honors program students and a similarly academically 
prepared sample of non-honors students, thus controlling for the 
widely recognized inputs that likely differentiate honors students at 
the point of matriculation in order to illustrate the value added of 
the honors program experience. The data collected here show hon- 
ors students outperforming the comparable non-honors group in 
measures of second-year retention and four- and five-year gradua- 
tion, regardless of pre-college academic preparation. The evidence 
suggests that the environment of the EKU Honors Program does 
have a positive effect on retention and graduation rates. The impact 


Ficure 6. RETENTION AND GRADUATION RATES FOR HIGHLY INVOLVED 
vs. Less INVOLVED Honors STUDENTS FOR 
Fatt 2010—Fatt 2012 INcomiNa First-YEAR STUDENTS 


916 96.4 
88.0 
82.8 
64.4 
55.7 
EKU EKU 


Honors Honors 
First-Year to First-Yearto Scholar Four-Year — Five-Year 
Second-Year Second-Year Graduation Graduation Graduation 
Retention _ Retention 


lm Highly Involved (n = 83) Less Involved (n = 174) 
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on a university's retention and graduation rates would be profound 
if more students were exposed to the honors program environment. 
In an era of public scrutiny and with the proliferation of perfor- 
mance-based funding (distribution of funding based on metrics 
such as retention and graduation rates, among others), making the 
case to high-level university administration that honors education 
positively impacts these metrics for its students is extremely benefi- 
cial for honors deans and directors. 

Additionally, a stark difference in simply participating exists 
between the EKU Honors Program and having high levels of 
involvement within the program. Students who participated in just 
one of the activities used to measure level of involvement had much 
higher rates of graduating as honors scholars and graduating from 
the university in four years than their less involved counterparts. 
This difference between being highly involved and less involved in 
honors activities suggests that providing meaningful opportunities 
for involvement creates an environment that positively affects the 
desired outcome of increasing graduation rates. 


LIMITATIONS 


A few limitations deserve some attention when considering 
this study. First, it may be the case that students who self-select into 
the honors program and choose to participate may be especially 
predisposed to the student success outcomes measured here. This 
predisposition may account for some of the gaps between the hon- 
ors group and the comparable non-honors group; while the groups 
have similar pre-college academic profiles, this study does not mea- 
sure the students’ attitudes toward education, the honors program, 
or the college as a whole. 

We also recognize that many of those students who fall into 
the category of less involved honors students are highly involved 
in other aspects of university life. The EKU Honors Program has 
had a long tradition of students who take leadership positions in 
a wide variety of campus activities, including student government, 
fraternity and sorority life, and athletics. That a student appears in 
the category of less involved within the honors context does not 
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imply that the student is not otherwise invested in campus life. 
Moreover, the majority of students do not become involved in one 
of the significant activities measured in this study until after their 
first year. The second-year retention rate of those highly involved 
honors students, compared to that of the less involved students, is 
less meaningful when we consider this fact. 

Additionally, the extremely high rates that honors students have 
for some of the outcomes measured here present some challenges 
in data analysis. Since the honors data here can be considered to be 
population data, honors rates could be used as null values in tests 
of hypotheses about non-honors rates. In the case, however, of an 
honors rate of 100 percent (e.g., the second-year retention rate for 
the honors first-year class of 2010), a standard test of significance 
is not possible, and for rates near 100 percent, large samples would 
be needed. Hence, we opted to use confidence intervals to estimate 
rates for high-achieving non-honors students and compare them to 
the population rates for the honors students. It is also worth not- 
ing that the population of high-achieving non-honors students is 
not well defined since the sample came from a mixture of students 
who decided not to enter the honors program, students who were 
enrolled in English 105, and students who had high ACT scores 
and high school GPAs; thus, it is not clear whether finite popula- 
tion correction factors might be needed since the population size 
is ambiguous. 

Finally, we recognize that the EKU Honors Program is in a 
unique position to send a large number of students each year to 
regional and national conferences because of its $1.8 million endow- 
ment designated for these purposes. Other measures of involvement 
or of high-impact practices that are distinctive to other programs 
may be better indicators of the value added of honors education at 
those institutions. 


CONCLUSION 


This study adds to the research on the value added of honors 
education by utilizing some of the core principles of Astin’s I-E-O 
model for longitudinal study of student success in higher education. 
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Looking quantitatively at the differences in outcomes between hon- 
ors and non-honors students, while controlling as much as possible 
for the inputs of these students on the basis of pre-college academic 
preparedness, the study shows a demonstrable difference in first- 
year to second-year retention and four- and five-year graduation 
rates between those students who participated in the EKU Honors 
Program and comparable students who did not participate in hon- 
ors. In addition, this study examines the differences in outcomes 
of those honors students who participated in a set of high-impact 
practices available in the EKU Honors Program. Being highly 
involved within the honors program correlates strongly to higher 
outcomes in persistence to the second year of college and four- and 
five-year graduation rates. In short, the environment of the EKU 
Honors Program positively impacts student outcomes and provides 
a significant added value not only for those students but also for 
the university as a whole. By creating an environment that leads 
to higher second-year retention rates and graduation rates for its 
students, honors education can raise these rates for institutions as 
a whole, making allocation of resources to honors education a sig- 
nificant and impactful strategic option for a university. 
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BACKGROUND 
Success and Equity 


De success is challenging. Yet schools and colleges across 
the country, indeed, around the world, seek to do it in order 
to demonstrate value. While we know that success depends upon a 
variety of skills that individuals develop into competencies, these 
can be difficult to measure in an academic setting. For exam- 
ple, as educators, we hope that success is an outcome of lifelong 
learning, but the measurement of lifelong learning requires sophis- 
ticated approaches that can be difficult to deploy across a broad 
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population (Riley and Claris 2008). As a result, administrators 
and instructors will often gravitate toward more readily available 
measures of success such as individual grades, grade point aver- 
ages (GPAs), or standardized test scores. While these measures can 
provide insight into performance in a particular setting, commonly 
a didactic instructional environment, they do not account for the 
variety of experiences that mold and shape an individual’s capacity 
for success. In fact, some educators might argue that these limited 
measures ignore some of the most important aspects of potential 
for success, such as, for example, resilience. 

One illustration of the lack of insight into student learning that 
grades are capable of providing can be found in the early devel- 
opment of the Force Concept Inventory (Halloun and Hestenes 
1985). This test is designed to determine how students understand 
motion and is typically employed to pretest this knowledge so that 
an instructor can tailor a class to meet the needs of the enrolled stu- 
dents. During the development of this test, it was administered to 
600 introductory physics students both before and after taking an 
introductory college physics course. Halloun and Hestenes (1985) 
found that students who received an A in the course were equally 
likely to have changed their understanding of motion after taking 
the course as students who received a C in the course. Thus, the 
students who earned an A did not necessarily understand motion 
better, but they were simply better at memorizing equations and 
plugging in values to get appropriate answers. The grade of A did 
not reflect their actual learning of the physical concepts, their 
knowledge, or their ability to apply this knowledge. 

College admissions programs commonly use high school GPA 
and standardized tests such as ACT and SAT to predict success 
in making admission decisions, but several studies show these 
to be, at best, moderate predictors of college GPA and retention 
(Anastasi 1963; Daugherty and Lane 1999; DeBerard, Spielmans, 
and Julka 2004; Galicki and McEwen 1989; Wolfe and Johnson 
1995). In terms of equity, the work of Banerji (2006) and others 
(National Research Council 1999) shows that standardized tests are 
biased against underrepresented minority and low socio-economic 
status populations. Thus, any effort to base admission on such a 
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test biases the admission standards against these groups. Interest- 
ingly, in a study of approximately 34,000 students from 30 colleges 
across the United States, Kobrin and Michel (2006) found that nei- 
ther the SAT nor the high school GPA were definitively predictive 
of the first-year college GPA. Most studies of this nature explore 
the potential correlation between GPAs or test scores at two dif- 
ferent times, spanning high school and college. While this can be 
instructive, we posit that college GPA remains a limited measure 
of a certain type of success and that this measure is not necessarily 
predictive of success in postgraduate endeavors. 

Weerheijm and Weerheijm (2012) provide a compelling argu- 
ment for the establishment of competency-based admission and 
performance standards that lead to the development of “excellent 
and successful professionals” (p. 229). In their survey of honors 
programs administered in a non-graded environment, they iden- 
tify three key factors that are most likely to produce “professional 
excellence” in graduates: personal characteristics, motivation, and 
study environment (239). Personal characteristics include intelli- 
gence, creative thinking, openness to experience, desire to learn, 
drive to excel, and persistence. They suggest that honors admission 
programs consider evidence of these factors as criteria for admis- 
sion. Motivation is perceived as a long-term construct: students 
who set long-term mastery goals for themselves are more likely to 
achieve educational success than students setting short-term per- 
formance goals. Fostering the development of these characteristics 
and motivation requires an environment that makes explicit the 
relevance of college learning to the workplace. Complementing this 
work, Mould and DeLoach (2017) encourage honors programs to 
identify program-specific measures of success that will lead to the 
identification of assessment tools aligned more directly with those 
measures. 

Honors programs provide a crucial opportunity for addressing 
equity in higher education. Astin (2016) suggests that the Ameri- 
can system of higher education inherently provides differential 
opportunities to students with differing levels of academic prepa- 
ration. He blames this inequity on higher education’s fascination 
with grades and standardized tests and the use of these metrics as 
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gatekeepers for access. By extension, limiting participation in hon- 
ors experiences in higher education to those with a high GPA or test 
score further disadvantages those who enter higher education at an 
already accumulated disadvantage. According to Kuh (2008) and 
Finley and McNair (2013), these are the very students who benefit 
the most from these types of engaging and productive experiences 
in college. Using NSSE data, Kuh revealed a generally positive rela- 
tionship between high-impact or engaged experiences, the types 
of experiences often offered through honors programs, and mea- 
sures of student learning and achievement. Interestingly, he found 
these effects were more pronounced for minority students and stu- 
dents with relatively low ACT scores. His results point to benefits 
of participation in these high-impact practices for all students, but 
especially for students from groups historically underrepresented 
in higher education and those least likely to have the opportunity 
to engage in them. 


A LIBERAL EDUCATION APPROACH TO STEM EDUCATION 


Michigan Technological University is a STEM-focused institu- 
tion where 95 percent of undergraduate students pursue degrees 
in a science, technology, engineering, or mathematics field. While 
STEM education is increasingly viewed as the solution for our 
nation’s economic decline (Olson and Riordan 2012) and our 
world’s most pressing social and environmental challenges (Beatty, 
Greenwood, and Linn 1999), considering how STEM education 
prepares undergraduates for the 21st century is important. In this 
rapidly changing world, we must cultivate the skills that will drive 
success and satisfaction: integrating knowledge across contexts, 
lifelong learning, intercultural effectiveness, and leadership. 

Common contemporary models of STEM undergraduate edu- 
cation focus on the delivery of content and assessment of learning 
via individual learning outcomes associated with specific products 
of the course environment (Olson and Riordan 2012). In some cases, 
schools and colleges reach beyond this environment to incorporate 
other learning or co-curricular contexts and assessment methods 
such as qualitative evaluation; however, adoption of these methods 
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is not widespread, and both program management and assess- 
ment can be time-consuming and costly (Sheppard, Macatangay, 
Colby, and Sullivan 2009). In addition, several high-profile STEM 
educators have called for the integration of liberal arts and STEM 
education, citing this integration as essential to the development of 
a competitive STEM workforce (e.g., the Annual Engineering and 
Liberal Education Symposium at Union College). 

The Wabash National Study of Liberal Arts Education describes 
seven liberal arts learning outcomes commonly associated with 
the development of wisdom and the responsibilities of citizenship: 
(1) integration of learning, (2) inclination to inquire and lifelong 
learning, (3) effective reasoning and problem solving, (4) moral 
character, (5) intercultural effectiveness, (6) leadership, and (7) 
well-being (King, Brown, Lindsay, and VanHecke 2007). Strikingly, 
these seven outcomes are interdependent, each contributing to the 
holistic development of the individual. Furthermore, each outcome 
is viewed as multidimensional: the achievement of each outcome 
requires integration of abilities across cognitive (what and how one 
knows), intrapersonal (who one is and one’s sense of identity), and 
interpersonal (how one relates with others) domains. For instance, 
consider how problem solving and leadership skills relate to each 
other and how both of these skill sets require maturity in intraper- 
sonal and interpersonal domains as well as the cognitive domain. 

The concurrent development of students across cognitive, 
intrapersonal, and interpersonal domains is described by the the- 
ory of self-authorship. Baxter Magolda (2008) provides a succinct 
description of self-authorship as “the internal capacity for an indi- 
vidual to define one’s beliefs, identity and social relations” (p. 269). 
This theory is rooted in the work of Kegan (1994), who argues that 
this development provides a necessary foundation for individuals 
to meet the expectations of adulthood. Baxter Magolda’s 21-year 
longitudinal study of young adults age 18 to 39 supports this claim 
(Baxter Magolda 2001). In this study, she found that participants’ 
roles and responsibilities required them to analyze data, explore 
and evaluate diverse perspectives, understand context and oth- 
ers’ frames of reference, and negotiate competing interests. Each 
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of these steps is useful for weighing alternatives and arriving at a 
judgment. Executing these tasks requires self-authorship to ensure 
that individuals are not overwhelmed by external influence, are 
confident in their ability to make defensible decisions, and are able 
to collaborate productively with colleagues. 

Specific examples of the need for self-authorship abound in 
society. For instance, in today’s global/social context, adults engage 
collaboratively with multiple diverse others. The development of 
productive relationships requires intercultural maturity, which 
depends on cognitive, intrapersonal, and interpersonal develop- 
ment. According to a 2007 report by the Association of American 
Colleges and Universities (AAC&U), industry increasingly expects 
higher education to encourage this development in undergradu- 
ate students, stressing teamwork, intercultural competence, and 
a greater emphasis on complex problem solving (AAC&U 2007). 
Indeed, higher education itself emphasizes social responsibility as a 
key outcome for addressing the challenges of the 21st century. 

Self-authorship requires the individual to shift from being 
uncritically dependent on external authorities for values, beliefs, 
identities, and loyalties to defining these elements internally. 
Individuals develop self-authorship when they are encouraged to 
construct and explain their views in learning environments that 
provide opportunities to explore alternative interpretations and 
that are emotionally supportive of the challenges of the knowledge- 
construction process (Baxter Magolda 2001; Kegan 1994; Pizzolato 
2005). Figure 1 presents a diagram of the levels of self-authorship. 
In the movement from “Following Formulas” to entering the 
“Crossroads,” individuals begin to experience and respond to ten- 
sions associated with continued reliance on external formulas as a 
means of defining themselves, their relationships, and their beliefs. 
As individuals move into the crossroads, they more openly question 
external authorities and begin to construct, listen to, and cultivate 
their internal voice. Once self-authored and ultimately internally 
defined, individuals trust the internal voice; build upon that foun- 
dation; and become secure in their identities, relationships, and 
beliefs. It is important to note that the development of self-author- 
ship is not a linear experience and that the course of development 
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rarely unfolds smoothly from one level or way of making meaning 
to the next. Rather, the developmental trajectory is punctuated with 
meanders, sprints, and setbacks. Nevertheless, identifiable mile- 
stones do exist. 

Without an intentional intervention, most undergraduate stu- 
dents—and even college graduates—define themselves through 
external formulas rather than self-authoring their beliefs (Baxter 
Magolda 1992, 2001; Baxter Magolda, King, Taylor, and Wake- 
field 2012; Belenky, Clinchy, Goldberger, and Tarule 1986; Kegan 
1994; King and Kitchener 1994; King and Mayhew 2002). Evi- 
dence shows, however, that with appropriate support this tendency 
can be changed. Several types of experiences produce higher 
degrees of self-authorship among undergraduates (King, Baxter 
Magolda, Barber, Brown, and Lindsay 2009; Barber, King, and 
Baxter Magolda 2013). These include experiencing dissonance in 
academic settings, being challenged to evaluate knowledge claims 
and take ownership of beliefs, encountering diverse perspectives, 
and addressing tragedy or complex personal relationships. Also 
essential is the identification of a community of support where pro- 
cessing of these challenging experiences occurs. Unfortunately, this 
demand often occurs post-graduation, leaving individuals to face 
significant challenges with insufficient preparation and potential 
risk to themselves, the people around them, and the organizations 
and systems they are trying to improve (Flores, Matkin, Burbach, 
Quinn, and Harding 2012). To foster the growth of self-authorship 
in an academic setting, a supportive environment can be created 
through what Hodge, Baxter Magolda, and Haynes (2009) refer to 
as the “Learning Partnership Model” 


LEARNING PARTNERSHIP MODEL 


Designed as a practical approach to transform both curricular 
and co-curricular learning, the learning partnership model (Baxter 
Magolda and King 2004) grows out of the theory of self-author- 
ship. To empower individuals to explore the complex landscape of 
knowledge, identities, and relationships, the learning partnership 
model incorporates three key principles: 
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1. Validating learners as knowers. Ensure that students know 
their voices are important and encourage them to share 
ideas and viewpoints while muting the voice of faculty as 
“the” authority, thus helping students to see the instructor as 
human, approachable, and concerned; 


2. Situating learning in learners’ own experience. Recognize and 
acknowledge that students bring their personal experiences 
into the classroom, explain the relevance of material to stu- 
dents’ daily lives, avoid marginalizing students, and provide 
opportunities for self-reflection to help students become 
clearer about what they know, why they hold their beliefs, 
and how they want to act on them; and 


3. Defining learning as mutually constructing meaning. Frame 
learning as something experienced together when both the 
instructor and the student share perspectives; students see 
that the instructor is continuing to learn through their work 
together and demonstrates lifelong learning. 


The key to a successful learning partnership is the balance 
of challenge and support necessary to push students toward self- 
authorship without triggering a reliance on old ways of constructing 
identity, relationships, and knowledge. Educators and administra- 
tors have used this model to design effective learning partnerships 
for learners in many situations, such as orientation programs, 
undergraduate courses, and internships. (Detailed examples can be 
found in Taylor, Baxter Magolda, and Haynes 2010; however, there 
is little evidence that this model is used much in the undergraduate 
STEM educational setting.) 


THE PAVLIS HONORS COLLEGE EDUCATIONAL FRAMEWORK 


The educational framework of the Pavlis Honors College at 
Michigan Technological University is designed to encourage the 
development of self-authorship by exposing students to a chal- 
lenging educational setting in a supportive learning environment. 
As students encounter and traverse the crossroads, the framework 
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reflects the levels of self-authorship that students should encounter, 
as well as the learning partnership necessary for this development. 
The Pavlis Honors College (PHC) framework is an adaptation of 
that proposed by Taylor and Haynes (2008) for the honors college 
at Miami University of Ohio. The framework articulates incoming 
student traits, developmental goals, student learning outcomes, 
faculty and staff expectations for engaging with students, and iden- 
tification of learning experiences where development is enabled. 
Table 1 summarizes the current framework for the first year of the 
program (year two for a traditionally enrolled college student). 

The program structure follows a tiered model of educational 
development associated with both the cognitive and affective 
domains of Bloom's taxonomy (Bloom, Engelhart, Furst, Hill, and 
Krathwohl 1956; Krathwohl, Bloom, and Masia 1973) and the self- 
authorship theory described above. The program integrates Baxter 
Magolda and King’s (2004) learning partnership model across 
three major elements: (1) a series of developmental seminars, (2) 
a set of required co-curricular activities with structured reflection, 
and (3) advising support. These elements provide opportunities 
for students to foster self-authorship: increasingly complex ways 
of making meaning about one’s identity, relationships, and beliefs. 
Students collaborate with faculty during seminars to explore con- 
cepts related to personal and social identities, cultural maturity, 
empathy, mindfulness, collaboration, and communication via 
dialogue. Students also define an academic enhancement (e.g., 
minor, certification), an immersion experience in which they 
apply their skills and knowledge in a new and unfamiliar context 
(e.g., an internship, international experience), an honors project 
that reflects their learning, and a leadership or mentorship activity. 
All program elements involve guided or semi-structured reflection 
with a faculty mentor designed to provide the students a platform 
through which to reflect on their learning and make meaning of 
their experiences and to encourage the development of self-author- 
ship. Figure 2 presents a diagram of the elements of the Honors 
Pathways Program. 


124 


GPA as PRODUCT 


PRELIMINARY ASSESSMENT 
Self-Authorship 


In order to determine if students are moving through the stages 
of self-authorship, a rubric was developed to score students’ reflec- 
tive essays. Specifically, the first and final reflections of Seminar I 
were scored to illuminate differences in the ways in which students 
make meaning of their experiences from the beginning to the end 
of one semester in the program after engaging with the honors col- 
lege curriculum. The following will explain the process for initially 
creating the rubric as well as how it was used to score students’ 
reflections throughout the course. 

The rubric went through several iterations before being used 
to score student responses to reflective prompts. The first stage of 
development was to align three prominent student development 
theories: “self-authorship,’ focusing on intrapersonal development 
(Barber and King 2014); “developmental trajectory of social justice 
allies,” focusing on interpersonal development (Waters 2010); and 
the “reflective judgment model,’ focusing on cognitive development 
(Kitchener and King 1990). Waters (2010) and Kitchener and King 
(1990) were incorporated because those frameworks gave a more 
focused picture of how students typically progress through the 
interpersonal and cognitive domains of development. Waters’ the- 
ory (2010) specifically focuses on how students relate to each other 
in diverse settings (interpersonal development), and Kitchener and 
King (1990) focus on the ways in which students make decisions 
(cognitive development). While self-authorship theory encompasses 
development in all three domains of development (interpersonal, 
intrapersonal, and cognitive), the other two theories served to better 
inform the developing rubric by giving myriad examples of student 
responses that indicate various levels of development. Incorporating 
these three frameworks into the rubric allowed for a more compre- 
hensive view of PHC student development throughout the semester. 

Each aforementioned theory has its own development scales, 
each organized into stages that represent various levels of develop- 
ment. As noted above, self-authorship theory has three stages: the 
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initial following formulas stage, the intermediate crossroads stage, 
and finally the self-authorship stage (Barber and King 2014). In 
similar fashion, the other two theories incorporate their own stage- 
style rubric, moving from less-developed to more-developed ways 
of thinking. Waters’ theory has three stages—initial, intermediate, 
and mature—and the reflective judgment model has seven stages 
that indicate increasingly mature and developed ways of decision 
making (Waters 2010; Kitchener and King 1990). Relying primarily 
on the self-authorship stages outlined by Barber and King (2014), 
we created an initial rubric and then tested it against the first week’s 
reflective responses. Quotations were selected from the first round 
of reflections and organized from least to most developed. This 
process revealed that a finer gradation of development was needed 
to capture smaller distinctions in student developmental trajecto- 
ries. Therefore, each level was expanded to include sub-levels that 
fully encompassed the nuanced differences in students’ methods of 
making meaning from their experiences. This process resulted in a 
nine-level progressive scale including three levels (early, mid, and 
advanced) within each of the three self-authorship stages. Seven of 
these levels were represented within the sample set. A description 
was included for each level that details the characteristics of student 
responses at each stage. The final iteration of this self-authorship 
rubric provided examples of student responses indicative of the 
various levels of development. Table 2 provides a summary of the 
rubric levels represented within the data set, characteristics sought 
in the reflections, and representative reflection quotations. 

Once the rubric had been finalized, it was then used to evaluate 
the honors college pilot cohort students’ first and last reflections of 
the semester. Specific quotations were chosen from each reflection 
that were indicative of a certain level in the rubric along the inter- 
personal, intrapersonal, and cognitive domains. Each student was 
given a score for each dimension of development, and scores from 
their initial reflection were then compared to those in their final 
reflection at the end of the semester. It is important to note that not 
all student responses included enough content for evaluation along 
all three dimensions; in these cases, students were given scores only 
for the dimensions that could be evaluated. 
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Grade Assessment 


Since our program does not consider GPA as a metric for admis- 
sion or retention, we were interested in understanding whether our 
applicants possessed a broader range of GPA than one might expect 
in an honors college. In other words, we wondered if we still were 
attracting students with high GPAs despite our goal of appealing to 
students from a range of academic performance levels. To answer 
these questions, we collected semester GPAs for our pilot cohort 
and calculated mean values and individual differences. 


RESULTS AND DISCUSSION 
Self-Authorship Assessment Results 


During the pilot year of the program, 31 students agreed to 
have their written reflections coded for self-authorship charac- 
teristics. Of these, 26 completed both reflection assignments. 
These students were second- through fourth-year students who 
self-selected into the honors college and enrolled in the first hon- 
ors seminar, which was a one-credit course designed specifically 
to advance self-authorship. Each written reflection was scored to 
indicate the level of self-authorship development within the inter- 
personal, intrapersonal, and cognitive domains. Figure 3 presents 
the results of applying the self-authorship rubric to the first and 
final reflections. Each graph shows the number of students coded 
into each developmental level for the first and final reflections over 
the three domains. As one moves vertically along the y-axis of each 
graph, the level of self-authorship becomes more advanced. The 
graphs show an overall shift of the distribution of the entire student 
population toward demonstrating higher levels of self-authorship 
(upwards) in all three dimensions over time. It is important to note 
that missing bars in these graphs indicate that no students in the 
data set fell into this level of development for this dimension. 

To reveal individual changes in self-authorship development 
within all three domains, we performed an individual analysis of 
each student in the pilot cohort. This analysis revealed three main 
categories of developmental change occurring in students over the 
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course of the semester. These three categories of change are shown 
in Table 3, where we share our coding for three different students. 

Subject #21 shows an early stage developmental trajectory: the 
student is still following formulas but shows some growth in one or 
more domains. Six students coded into this category. At this devel- 
opmental stage, students are resisting challenges to their externally 
defined self-concept. One student in this stage of development 
wrote in the final reflection: 


I wouldnt say that I see myself, my goals, and my success 
differently. I am pretty firm in those beliefs, although I have 
enjoyed exploring these topics more in depth. (Subject #8) 


Another student found little value in the reflections: 


It was very frustrating to do the weekly reflections, because 
I dont really feel that I got anything out of it. I tried to really 
consider the questions and dig deep to answer them, but I 
still don’t really feel like I got much out of them. (Subject #5) 


A second developmental trajectory revealed students actively 
encountering the boundary between following formulas and self- 
determination (see Subject #1, for example). For these students, the 
uncertainty of defining oneself creates a significant barrier that is 
difficult to overcome. There were 10 students who fell into this tra- 
jectory. One student marvels at the development of self-awareness: 


As far as how I look at myself, I am a little more critical of 
my own views and my own contributions. I have learned to 
take a step back and actually think about my views, what 
the motivations are behind those views, and how to analyze 
and learn from past experiences. (Subject #18) 


A second student in this category reflects on learning to withhold 
judgment: 


This class has also made me better at letting others show me 
who they are rather than to just pick an identity for them 
based on what they look like. (Subject #16) 
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In the third category of developmental trajectory, 10 students 
exhibited significant growth in self-authorship, as exemplified by 
Subject #15. Here, students are advancing two or more stages in 
at least two domains. Most of these students are demonstrating 
thought processes consistent with the mid- to advanced-crossroads 
stages. In a final reflection, one student wrote about learning to 
construct a new worldview: 


I learned how to better suspend judgement and look at all 
different sides before forming an opinion. I also learned to 
take into account the lens that I look at the world through 
in my everyday life. The lenses can consist of all of the expe- 
riences, values, and ideas that you have about the world. 
Overall learning to have a more balanced opinion and tak- 
ing time to learn about other points of view has made mea 
better person and that these experiences will help me sig- 
nificantly in the future. (Subject #22) 


Another student reflected on discovering being externally defined 
and found value in developing more self-awareness: 


Before this course I had never really tried to define my own 
personal values, instead I just accepted a mold of other val- 
ues that had been impressed on me. After contemplation I 
realized that while some of these values are true to me there 
are also some that don’t apply to me as I thought they had. I 
also learned that I have other values that I hadn't previously 
considered. This is important to learn as early as you can, 
as well as to acknowledge that they are dynamic and can 
change based on experiences therefore it is an important 
activity to do periodically. (Subject #23) 


Overall, when assessed in this manner, the majority of students 
in this pilot study demonstrated higher levels of self-authorship in 
their final reflection as compared to their first reflection. There was 
little difference based on year in college, with second-year students 
showing a distribution of developmental trajectories similar to 
third- and fourth-year students. 
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Grade Assessment Results 


Among our 31-student pilot cohort, the average student GPA 
in the semester of application to the honors college was 3.55 out of 
4.00 with a median of 3.69 and a range from 2.12 to 4.00. This dis- 
tribution is skewed with the weight of scores toward higher GPAs. If 
we had applied a cutoff GPA of 3.50, seven of these students would 
not have been admitted to the honors college. By the end of the first 
seminar, these same students exhibited a mean semester GPA of 
3.61, median of 3.66, and a range of 2.76 to 4.00. For each student, 
we calculated the difference between the GPA during the semester 
of enrollment in the first seminar (enrollment semester) and the 
GPA during the prior college semester when the student applied 
for admission to the honors college (application semester). Table 4 
compares these GPAs averaged for groups of students sorted by GPA 
quartile. Among the top three GPA quartiles, we see a small down- 
ward shift in GPA, less than or equal to 0.18. For these students, 
the downward shift is sufficiently small such that they maintain an 
average GPA of over 3.50. Interestingly, however, students in the 
lowest quartile demonstrate an average increase in GPA of 0.28. 
Thus, while students with high GPAs continued to maintain high 
GPAs, those students at the greatest risk for not being admitted to 
an honors program demonstrated significant gains in GPA while 
exposed to an environment designed to advance self-authorship. 

At the end of the first seminar, six students had a semester 
GPA below 3.50. Two of these students experienced an academic 


Table 4. SemesteR GPA CHances BY QuanrriLe 


GPA Average 
Semester of Nona 
Application to Enrollment in 
Quartile GPARange  HonorsCollege —‘FirstSeminar _Difference 


4.00 
3,71 to 3.99 
3.44 to 3.70 

< 3.44 
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setback pushing them below this threshold; the other four were on 
an upward trajectory. This analysis reveals that some of our high- 
est GPA students can experience individual setbacks in any given 
semester while some of our lowest GPA students can exhibit dra- 
matic increases in their individual GPA. The consequences of these 
shifts can be disastrous for students in a program that institutes 
a GPA cutoff for retention. If we had placed a GPA threshold on 
the program, only 24 out of 31 students would have been admitted 
and 2 of those 31 students would have been asked to leave after the 
first semester. This dismissal would have occurred without consid- 
eration of their demonstrated learning related to the key outcomes 
of self-authorship. 


Combining Data Sets 


To examine the relationship between self-authorship develop- 
ment and academic achievement as expressed by grades, we first 
explored the relationship between incoming levels of self-author- 
ship and academic achievement. We ranked all students in the 
cohort by their semester GPA upon application to the honors col- 
lege as well as by their demonstrated level of self-authorship across 
the three domains. A Spearman correlation (r = .24) of data revealed 
little to no relationship between GPA and level of self-authorship 
development. 

To examine how academic achievement might be related to 
self-authorship development, we summed the developmental levels 
of change across all three dimensions of self-authorship for each 
student. In Table 3, we have provided examples of the summed 
developmental stages calculated for each subject. For example, 
Subject #21 advanced one level—from mid following formulas to 
advanced following formulas—in the cognitive domain, did not 
advance in the intrapersonal domain, and advanced one level in 
the interpersonal domain. The resultant developmental level of 
change for this individual is the sum of these three values: two. 
We then categorized our participants by GPA quartile and identi- 
fied the associated percentage of students who had demonstrated 
no growth (still following formulas), some growth (entering the 
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crossroads), and significant growth (approaching self-authorship). 
The results are presented in Figure 4. This analysis shows that stu- 
dents of any GPA can achieve the highest level of self-authorship 
development, or the lowest. 

Complementing these results, we calculated the average dem- 
onstrated levels of advancement in self-authorship for students 
in each GPA quartile. Results are presented in Figure 5. We find 
that students in the lowest GPA quartile exhibit the highest aver- 
age growth in self-authorship, while students in the third quartile 
exhibit the lowest. It is interesting to note that students in the low- 
est GPA quartile also exhibit the largest increase in GPA from their 
application semester to the end of the first honors seminar, while 
students in the third quartile exhibit the largest mean decrease in 


FicureE 4. PERCENTAGE OF STUDENTS DEMONSTRATING THREE DIFFERING 
DEVELOPMENTAL TRAJECTORIES BY GPA QuanrrILe 
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GPA. It may be that the challenges that students in the third quartile 
were facing in terms of their academics were presenting a barrier to 
non-cognitive development; however, this assertion would require 
further study. 

A Spearman correlation of the individually ranked GPA and 
demonstrated overall change of level in self-authorship develop- 
ment (r = -.54) suggests a moderately negative relationship such 
that a higher GPA correlates to lower demonstrated self-authorship 
development. Thus, GPA is not a clear measure of learning in the 
context of our honors college learning goals. 

Just as Halloun and Hestenes (1985) found that “A” and “C” 
students were equally likely to have changed their understanding 
of motion after taking introductory physics, we find that some of 
our top GPA students lack development in self-authorship, while 
some of our lower GPA students exhibit high levels of develop- 
ment. Since we believe that development of self-authorship is a key 
to post-graduate success, our data suggest that GPA is not a clear 


Ficure 5. AVERAGE STUDENT INCREASE IN SELF-AUTHORSHIP BY STAGE 
SUMMED ACROSS THREE DIMENSIONS BY GPA QuartTILE 
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indicator of future self-authorship development or, by extension, 
post-graduate success. 


LIMITATIONS AND FUTURE WORK 


Reliable assessment of self-authorship is typically conducted 
through the use of an interview protocol specifically developed 
for this purpose (Baxter Magolda and King 2012). While many 
researchers have attempted to identify alternative methods for 
self-authorship assessment, none has proven to be as robust as 
the interview. In developing the protocol for this assessment, we 
consulted with Patricia M. King, an expert in self-authorship, who 
suggested a potentially effective alternative: assessment of student 
reflections in answer to prompts specifically designed to elicit 
responses addressing each of the three domains of development. 
Thus, our results are limited by the use of a new and as yet unvali- 
dated method of assessment. Despite this limitation, we were able 
to identify developmental stages for most students in the cohort 
who completed both the first and final reflections. Future work on 
the use of a written reflection protocol should include a thorough 
comparison of this new protocol with the accepted self-authorship 
interview protocol and refinement of the reflection prompts to 
assure that the reflections elicit from participants a well-rounded 
and thorough discussion of their level of development across all 
three domains. 

As a pilot study designed to provide insight for the planning 
and development of a new honors college, the study has a low num- 
ber of participants. Further, results are not compared to a control 
group who did not enroll in the honors seminar. In addition, the 
participants self-selected into the program, making them an excep- 
tional group for whom the messaging of the college resonated 
and for whom one might expect to see development. As the hon- 
ors college continues to grow, new students will be added to this 
assessment program, thus increasing the number of participants. 
In this study we used a pre- and post-assessment to study individ- 
ual development. To learn if the honors college is truly making a 
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contribution to self-authorship development among undergradu- 
ates, we will need to add to the study a set of students who do not 
enroll in the honors college but exhibit similar characteristics to 
those of our students, including those characteristics known to 
affect self-authorship development such as gender, race-ethnicity, 
and age. 


CONCLUSIONS 


Self-authorship development has been shown to produce gradu- 
ates who are better prepared to manage adversity and change, make 
meaningful decisions, benefit from their educational experiences, 
and learn deeply throughout their adult lives. Yet college students 
in the United States rarely advance beyond following formulas to 
the crossroads (Barber and King 2014; Baxter Magolda 2007, 2014). 
In our pilot study, we found that a focus on the learning partner- 
ship model in our courses correlates with a shift among a majority 
of our students to higher demonstrated levels of self-authorship in 
one semester. 

This pilot study also offered promising results indicating that 
GPA is nota strong measure of learning in the context of self-author- 
ship development. In fact, the GPA for this cohort was moderately 
negatively correlated with demonstrated level of self-authorship 
development, and students of all GPA levels demonstrated a vari- 
ety of levels and development of self-awareness (intrapersonal 
domain), relationship development (interpersonal domain), and 
knowledge construction (cognitive domain). This study also offered 
insight into the potential for a written reflection protocol to be used 
as an assessment for self-authorship. While more work is needed, 
the results shown here suggest that focusing our honors college on 
specific learning goals and using these as measures of success other 
than GPA provide a framework for our curriculum and assessment 
and also create an environment in which students may find a deeper 
connection between their self-defined future and their coursework 
such that GPA becomes a product of engagement with the honors 
college rather than a measure of potential for success. 
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As honors programs and honors colleges evolve and develop 
to become more diverse and inclusive, there is significant value in 
identifying learning goals based on educational theory and practice 
rather than relying on screening processes that employ metrics that 
place many promising students at a disadvantage. Theory and sup- 
porting practices can be used to guide admission policies, learning 
goals, instructional approaches, and assessment tools that create a 
welcoming environment for a diverse student body and encourage 
development of competencies that prepare students not only for 
work in their field of interest but for life in the 21st century. 
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Palen that take place early in students’ college career can 
strongly influence their academic engagement and success. 
Two experiences that honors programs may provide during the 
initial phases of the undergraduate experience are pre- or early- 
semester programs and honors residence halls. This study compares 
honors students who lived in an honors residence hall and/or took 
part in a pre-semester academic, credit-bearing class upon entry 
into college to their honors peers who did not elect these options. 
It tracks the degree of the students’ subsequent engagement with 
the honors program and also several measures of their academic 
success, such as grade point average (GPA), during their under- 
graduate experience. Results indicate that students who elected 
to participate in a pre-semester class and live in an honors resi- 
dence were more engaged in the honors program and had greater 
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academic success overall at the university than honors students 
who did not. This direct comparison of honors program students 
who have elected certain honors experiences to those who have not 
strengthens the claim that specific honors experiences add value to 
the undergraduate experience. 


RELATED RESEARCH 


Not surprisingly, the scholarship on first-year experiences and 
how they affect student success is voluminous: there are many arti- 
cles and books on the topic as well as the presence of a national 
center, the National Resource Center for the First-Year Experi- 
ence and Students in Transition, with a journal, monograph series, 
and annual conference dedicated to the topic. Yet, even with the 
prevalence of first-year experiences, research on the effects of hon- 
ors first-year seminars is limited and consists mostly of qualitative 
descriptions of local versions of the seminar. (See Vander Zee et al. 
2016 for a survey of the research on honors first-year seminars.) 
Moreover, the pre-semester honors academic experience, which is 
the focus of this study, warrants distinct treatment from a first-year 
seminar in that its time and length of delivery (i.e., pre-semester 
and four days) significantly influence its effects. 

We could find only one recent data-based study on a pre- 
semester experience (Perrine and Spain 2008) related to our study. 
Although Perrine and Spain (2008) did not specifically examine 
pre-semester experiences among honors students, the pre-semester 
experience they did evaluate was similar enough to the pre-semes- 
ter class for entering honors students that we examined to inform 
our thinking. Perrine and Spain (2008) did a two-year longitudi- 
nal study on the effects that an optional, non-credit, six-day-long, 
pre-semester orientation program had on academic credits earned, 
GPA, and college retention. The orientation program was designed 
to help incoming students integrate into the university community. 
It included speakers; workshops on academic and social issues; 
and social events involving students, faculty, and staff. Participants 
moved into campus residence halls one week prior to the begin- 
ning of the fall semester, and the orientation took place that week. 
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Enrollment was offered to all entering students, including transfer 
students. Perrine and Spain (2008) used multiple regression tech- 
niques to filter out other possible predictors of academic success, 
including high school GPA, ACT composite, and gender. They 
evaluated data from a student survey and found that although par- 
ticipants in their pre-semester orientation program indicated that 
the experience helped with their academic and social adjustment 
to college, the orientation program had little effect on retention, 
credits earned, and college GPA. 

As with first-year seminars, much scholarship exists on the 
effects of residence halls and living-learning communities on the 
success of students (for a selective survey see Frost and Kay 2015; 
also Rinn 2004). Little comprehensive data have been collected, 
however, specifically on the effects of the honors residence hall expe- 
rience on students’ academic outcomes (Rinn and Plucker 2004). 
As Rinn and Plucker (2004) note, how separating honors students 
from the general population via honors housing affects them “has 
not been studied comprehensively,’ and thus the research currently 
does not give insight into its possible benefits or repercussions (63). 

We found four older data-driven studies specifically on the 
effects of housing for high-ability students. DeCoster in two stud- 
ies (1966 and 1968) tracked the effects on academic achievement 
of placement of high-ability students in different concentrations 
in residence halls as compared to high-ability students randomly 
assigned to their residence halls. He found that a 50% to 100% con- 
centration of high-ability students living in the same residence hall 
produced a significant rise in GPA, whereas high-ability students 
randomly assigned a residence hall showed no significant increase 
in GPA. 

Another study (Duncan and Stoner 1975), which was under- 
taken at Southern Illinois University at Carbondale, compared the 
GPAs of 93 high-ability students, termed President's Scholars, who 
lived in the same residence hall, Smith Hall, over a period of three 
quarters to the GPAs of 84 other President's Scholars who were 
selected at random among those who lived elsewhere (other res- 
idence halls, off-campus, or with their parents at home). Results 


153 


SPISAK, KIRBY, AND JOHNSON 


were that the mean GPA of President's Scholars living in Smith 
Hall was not significantly higher than that of President’s Scholars 
living elsewhere. Yet, Duncan and Stoner (1975), on the strength 
of slightly higher GPAs for those in Smith Hall and their survey 
results, state that “there appear to be some positive effects on grade 
point averages” (7). 

In one other older study, Stewart (1980) compared the academic 
achievement of honors students in an honors residence hall to that 
of honors students living in non-honors residence halls. The results 
of his study indicate that “being a resident of the general honors 
unit is not a significant factor with respect to an honors student's 
GPA” (28). Stewart's study (1980), however, is limited in its scope. 
He tracked 74 honors students (30 who lived in honors housing 
and 44 who lived in non-honors housing) for only two semesters. 
The small sample size and short duration of Stewart’s study may 
make his results inconclusive. 

A more recent and comprehensive data-driven study that tracks 
the effects of an honors residence hall on the academic success of 
honors students was done by Campbell and Fuqua (2008). Their 
study tracked for a period of five years a cohort of 336 entering 
freshmen who were part of an honors program at a large, Mid- 
western, public university. The purpose of the study was to identify 
factors that were potential predictors of completion in the honors 
program. Their classifications were completers, partial completers, 
and non-completers. Campbell and Fuqua (2008) found that the 
most important discriminating variables were high school GPA, 
high school class rank, first-semester college GPA, and initial hous- 
ing assignment into either honors or non-honors housing. The 
study indicates that the first-semester college GPA was the most 
important predictor of completion in the honors program, but the 
second most important predictor of honors completion was hon- 
ors housing. Specifically, students who lived in honors housing for 
their first semester completed the honors program at a substantially 
higher rate than those who did not. What is not evident from the 
study, however, is whether this result was because the environment 
in honors housing helped with program completion or because 
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students who were initially more committed to the program chose 
honors housing. 

Finally, as part of a literature review of studies on the effects of 
housing for the general student population, Rinn (2004) considers 
the academic and social effects of living specifically in honors resi- 
dence halls. Based on her review of the literature, she speculates that 
honors students who live together in the same residence hall are 
“likely to facilitate and reinforce the academic achievement of one 
another” (70). Yet, in her conclusion she maintains that although 
living in an honors residence hall can influence or enhance aca- 
demic achievement, “the social effects are arguably controversial” 
(76). Rinn (2004) mentions as examples of possible negative effects 
self-segregation, the formation of “narrow peer groups,’ and experi- 
encing “isolation from the rest of the campus” (76). She thus leaves 
readers uncertain about the overall benefit of honors housing. 


RESEARCH METHODS AND SAMPLE 
Setting for the Study 


The Honors Program at the University of Iowa, which is a large, 
public, highly active research university, was founded in 1958 as 
a program within the College of Liberal Arts and Sciences. The 
honors program became university wide, serving all undergradu- 
ate degree-granting colleges, in 2006. Its student population today 
is about 3,200, with approximately 700 entering first-year students 
per year. Throughout its existence until 2013, the program's pri- 
mary focus was disciplinary honors, with students earning honors 
in the major at graduation by meeting departmental requirements 
and maintaining a strong grade point average at the university. 
In 2013, the honors program implemented a curriculum and 
program of study for awarding what then became known as “uni- 
versity honors.’ Many honors students (about 60%) still completed 
departmental honors as part of university honors, but either form 
of honors could be done separately. 

Through the years a variety of efforts have been directed toward 
building community among University of Iowa honors students 
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early in their university experience. The first of these was hon- 
ors-specific housing, largely in a residence hall known as Honors 
House, and the second, which was added later, was a pre-semester 
academic experience for entering honors students, titled “Honors 
Primetime.” This study looks specifically at these two opportunities 
for the value they add to the honors experience. 

In recent years, Honors House, an interdisciplinary living 
community for entering honors students, is the home for about 
one third to one half of the honors program entering class. It is 
specifically for entering honors students, and the only returning 
students who live there are the resident assistants. All resident 
assistants are honors students, but they are selected by the office of 
university housing and not by the honors program. Honors House 
is directly attached to the Blank Honors Center, which is home to 
the honors professional staff, classrooms, the Belin-Blank Interna- 
tional Center for Gifted Education and Talent Development, and 
a 12,000-square-foot student center open to all honors students. 
While the student center is open to all honors students, it is com- 
monly viewed, however, as the extended lounge space for Honors 
House by its residents. Although Honors House is commonly 
referred to on campus as a living-learning community, it does not 
have some of the features associated with these communities, such 
as common coursework and a shared disciplinary theme. Both the 
honors program and the Honors House resident assistants host a 
variety of events each year. Examples of these would be a scavenger 
hunt in the Blank Honors Center to meet our staff, a star-gazing 
event with an astronomy professor, an off-campus movie night with 
an invited faculty speaker, and an end-of-year gala organized by 
the students. None of these events is required, and participation 
varies between 10 and 50 percent of the Honors House residents. 
Overall, a strong sense of community exists within the residents of 
Honors House, and they are more likely to interact with the honors 
program professional staff than honors students who live in other 
residence halls. 

Offered the week before fall semester classes begin, the pre- 
semester academic experience, Honors Primetime, is a four-day, 
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one-credit-hour course divided into small academic workshops. It 
is an elective option for entering honors students, and about one 
third to one half of the entering class chooses to participate in any 
given year. Registration for Honors Primetime is done as part of 
the fall schedule; students identify topical areas that interest them, 
such as social sciences or public policy, rather than selecting a 
specific workshop. The workshops are capped at twenty students 
and vary in disciplinary topics. Some examples of topics include 
learning how flight developed in birds, the mechanisms of volcanic 
eruptions, food sourcing for local restaurants, and a choral group 
learning about protest songs. All workshops are hands-on; there 
is no homework outside of the workshop class times, which are 
morning and afternoons for three-and-a-half days; and no tests are 
allowed. Primetime begins with a welcome event and guest speaker, 
and it culminates with three- to five-minute presentations by 
selected students from each workshop to all the Honors Primetime 
participants. Grading is done on a satisfactory/unsatisfactory basis. 

Enrollment in Honors Primetime and selection of Honors 
House are optional for entering honors students. Honors Prime- 
time has about 400-500 students taking part each year, and Honors 
House has about 320-350 residents each year. Honors Primetime 
is open to any entering honors student who chooses to take part. 
Honors House is available on a first-come, first-served basis when 
honors students make their housing selections with the university. 
Interested students have never been closed out of Honors Prime- 
time if they registered on time, but some entering students have 
not been able to live in Honors House because they signed up for 
housing at a date later than other entering students. The majority of 
students who elect to take part in Honors Primetime also elect to 
live in Honors House. 


Preliminary Study 


This study was initiated to determine whether membership in 
the University of Iowa Honors Program affected student success 
and, more specifically, how first-year experiences influence success. 
A preliminary study compared the differences in the mean GPA of 
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students who entered the university as first-time, full-time, first- 
year undergraduates and honors members to those who qualified 
for entry in the honors program but did not accept membership 
in the program. Specifically, this group, which we call “honors 
peers,’ was defined as any student who entered the university with 
the ACT composite (230) and high school GPA (23.8) required for 
honors membership as an incoming first-year student but did not 
accept membership into the honors program. The requisite ACT 
composite and high school GPA were the only requirements for 
honors membership: that is, there were no additional application 
requirements other than accepting membership in the honors pro- 
gram. (The University of Iowa Honors Program invites qualified 
students after they are admitted to the university.) 

The data set of this preliminary study began with 4,300 stu- 
dents who entered the university as first-time, full-time, first-year 
undergraduate college students in fall semesters between 2013 and 
2016. From the original group, 175 cases were rejected for incom- 
plete information, which left 4,125. Of that number, 3,332 entered as 
members of the honors program, and 793 declined the honors invi- 
tation and entered the university as honors peers; we measured the 
GPA of these 4,125 cases after their first year of enrollment. When 
measuring the GPAs of graduates, we only considered the students 
who earned a bachelor’s degree at the university through the spring 
2017 semester. This cohort of graduates included 578 students who 
entered as honors members and 164 who were honors peers. 

The academic success, as indicated by college GPAs, of students 
who entered as honors members and those who entered as hon- 
ors peers (i.e., students of comparable academic ability) differed 
significantly (p < .05). As entering first-year students, the honors 
members had an average four-point high school GPA of 4.06 and 
an average ACT composite of 30.0, while the honors peers had an 
average four-point high school GPA of 4.03 and an average ACT 
composite of 29.3. In contrast, the honors members had higher 
GPAs at the end of the first year (3.46 versus 3.34) and at the time 
of graduation (3.52 versus 3.46) than the comparison group of hon- 
ors peers (p < .05 in a two-tailed test). That difference suggests that 
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specific components of the honors program had a positive impact 
on the students’ academic accomplishments. This possibility led to 
further research that has become the basis for the main study pre- 
sented herein, with the goal of determining which components of 
the honors program led to an increase in academic success. 


Main Study Design 


The sample for the main study was the same group of 3,332 stu- 
dents from the preliminary study who accepted membership in the 
honors program in the fall semesters of 2013 through 2016. Enter- 
ing honors students fell into four different groups: (1) students who 
lived in first-year honors housing; (2) students who participated in 
the Honors Primetime program; (3) students who did both; and 
(4) honors students who did neither. We then compared these four 
to one another on four separate outcome measures of academic 
success: (1) grade point at the end of the first academic year; (2) 
completion of 12 or more hours of honors coursework;! (3) com- 
pletion of the honors program requirements and University Honors 
graduation; and (4) GPA at graduation. Only the fall 2013-fall 2015 
entering honors cohorts (n = 2,610) were evaluated for the 12-hour 
completion outcome. 

To evaluate the effects of honors housing and Honors Primetime 
on the two GPA variables, we used hierarchical linear regression 
analysis in order to determine which variables have the greatest 
effects. To evaluate the effects on the two binary variables (where 0 
= did not complete the coursework or did not graduate with honors, 
respectively) we used logistic regression. These analyses control for 
the variables considered when looking at GPA outcomes: sex, first- 
generation status, ACT composite (or converted SAT score), high 
school GPA (four-point scale), and the student's college at entry. 
Since the data were current through the end of the spring 2017 
semester, we used only the fall 2013 through fall 2015 cohorts (n 
= 2,604). Awarding of university honors occurs at graduation. This 
fact limited the cases that could be used to just those students who 
started in fall 2013 and had earned an undergraduate degree at the 
university by spring 2017 (n = 578). 
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Additional factors, such as which college a student entered, 
are likely to influence these outcomes. We therefore also included 
variables that could be accessed for the whole sample to deter- 
mine whether these first-year experiences had unique effects on 
an outcome after controlling for other possible correlates. We used 
multiple linear regression to control for first-generation status (a 
binary variable where 0 = not a first-generation student), sex (a 
binary variable where 0 = male), ACT composite score, high school 
four-point GPA, and the student’s primary college in the first year 
(measured as a set of five dummy variables). All four cohorts were 
analyzed for GPA after the first year; only those students who had 
graduated by spring 2017 were analyzed for GPA at graduation. 

We ran regression models for each of the two GPA outcomes 
by adding in the binary independent variables of interest: honors 
residence, Primetime participation, and students who participated 
in both opportunities (where 0 = did not participate/apply for each 
of the three variables). Because of collinearity, which occurs when 
independent variables show too much correlation, isolating unique 
effects in statistical models can be difficult. To mitigate this con- 
cern, we used two models for each of the two GPA outcomes: one 
model with honors residence and Primetime participation present 
as distinct binary variables, and one model with the binary variable 
indicating the presence of both opportunities. The former model's 
variables could include students who did either one of the first-year 
experiences as well as students who did both, while the latter model 
includes a variable that only indicates participating in both first- 
year experiences. 

We also collected qualitative feedback from Primetime partici- 
pants via a survey. Specifically, we sent a survey to each participant 
within two weeks of the completion of the course. This survey 
included questions to gauge satisfaction with the specific workshop 
and instructor a student was assigned to as well as general questions 
about the Primetime experience and programming. The comple- 
tion rate of the survey was very high: an average of 80 percent of 
Primetime participants across the years completed the survey. 
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RESULTS 


The results suggested that honors housing and Honors Prime- 
time did affect the academic performance of the honors students. 
Figure 1 shows the average GPAs of the four distinct honors cohorts 
at the end of the first academic year. Students who took part in 
Primetime but did not reside in honors housing and students 
who lived in honors housing but did not participate in Primetime 
had slightly higher GPAs at the end of their first year than honors 
students who did neither. The students who had the strongest aca- 
demic start were those who both took part in Primetime and lived 
in honors housing. Specifically, they had significantly higher GPAs 
at the end of the first year than students who only took part in Pri- 
metime or only lived in honors housing as well as higher GPAs than 


Ficure 1. GRrADE Point AVERAGE AT THE END OF THE First ACADEMIC 
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students who did neither (F = 16.442 and p < .05). Further testing 
using Tukey post hoc tests revealed the “both” variable as the only 
cohort to differ significantly (p < .05). 

Progress in completing the honors coursework requirement of 
12 semester hours in the first four semesters also varied among the 
four honors cohorts. Figure 2 presents the percent of students meet- 
ing this milestone for each of the four groups. A greater percentage 
of students who took part in Primetime or lived in honors hous- 
ing completed the honors coursework requirement than students 
who did neither experience. The effects, however, of honors hous- 
ing or Primetime as isolated variables are not significant (chi-square 
tests showed no significant association). About half of the students 
who either took part in Primetime or lived in honors housing com- 
pleted the honors coursework requirement. The cohort that stood 


Figure 2. PERCENT OF Honors STUDENTS MEETING THE COURSEWORK 
REQUIREMENT FOR University Honors 
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out once again and was significantly more successful than all other 
cohorts was the one in which the students took part in Primetime 
and lived in honors housing (p < .05; a chi-square evaluating the 
“neither” group demonstrated significance in a negative correlation 
with completing the honors coursework requirement, with p < .05). 
Over 65 percent of this cohort completed their honors coursework 
requirement, which is a necessary step in graduating with univer- 
sity honors. This represents a nearly 30 percentage point increase 
in completing the coursework requirement over students who nei- 
ther took part in Primetime nor lived in honors housing. Indeed, 
honors students who participated in both Primetime and lived in 
honors housing were almost twice as likely to complete their honors 
coursework requirement. As with first-year GPA, participation in 
both honors experiences markedly affects student academic success. 

Participation in both Primetime and honors housing also had 
significant effects on graduation with University Honors (see Fig- 
ure 3). A pattern similar to that in Figures 1 and 2 is apparent. Less 
than one-fourth of the honors students who neither took part in 
Primetime nor lived in honors housing graduated with University 
Honors by spring 2017. Students who took part in Primetime or 
lived in honors housing graduated with University Honors at about 
a7 percentage point higher rate, which appears slightly higher but 
is not significantly different than the students who did neither. The 
honors cohort that was most successful at completing university 
honors elected both to take part in Primetime and live in honors 
housing: over 40 percent of that cohort graduated with Univer- 
sity Honors, which was significantly greater than the three other 
cohorts (p < .05). 

Finally, Figure 4 presents results comparing differences in GPA 
at graduation. Although fewer than half of the honors students in 
all four cohorts graduated with university honors, they were all 
successful in regard to their GPAs at graduation; the average GPA 
was 3.5 or better for each group. There was no significant difference 
among the four cohorts. 

As noted previously, we used regression statistical analysis to 
look at the influence of attributes that were intrinsic to the student 
population, such as sex, ACT score, high school GPA, whether their 
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parents had earned college degrees, and what college the students 
entered at matriculation to the university. The three scenarios for 
the first-year experiences were students who participated in Hon- 
ors Primetime (n = 1,754; n = 273 for graduation data points); 
students who were in honors housing (n = 1,205; n = 207 for data 
points collected at graduation); and students who both participated 
in Honors Primetime and lived in first-year honors housing (n = 
811; n = 123 for graduates). We employed regression analysis for 
each of the four outcomes under examination: GPA at the end of 
the first year; completion of 12 semester hours of honors course- 
work; GPA at graduation; and graduation with University Honors. 

Tables 1 and 2 present a regression model of factors influenc- 
ing first-year GPA. They show a significant and positive association 
between first-year GPA and honors residence, female sex, ACT 


FicureE 3. PERCENT OF Honors STUDENTS COMPLETING UNIVERSITY 
Honors IN TIME FOR GRADUATION 
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composite score, and high school four-point GPA (p < .01). There is 
a negative correlation with first-year GPA for first-generation status 
and membership in one of the undergraduate colleges relative to 
the omitted college we used as a reference group in regression mod- 
els. Participation in Primetime does not demonstrate a statistically 
significant association (Table 1). Comparable effects are apparent in 
the model using the binary variable indicating participation in both 
Primetime and honors housing (Table 2), with positive correlations 
for the Primetime and housing combination while controlling for 
sex, ACT, and high school GPA, and negative correlations for one 
of the colleges and first-generation status. 

Next, we looked at GPA at graduation for those honors students 
who completed a bachelor’s degree at the university by spring 2017. 
Tables 3 and 4 present the results of this analysis. By the time of 


Ficure 4. GRADE Point AVERAGES AT GRADUATION FOR THE Four 
Honors Conorts 
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Note: The 3.33 baseline for the graph represents the minimum GPA for honors program membership. 
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graduation, only high school four-point GPA and ACT composite 
have a significant positive effect on college GPA. First-generation 
status and membership in one of the undergraduate colleges are 
negatively correlated with GPA at graduation. Neither honors 
housing nor Primetime participation has a significant impact on 
GPA at graduation (Table 3). 

The combination of both Primetime and honors housing did 
not demonstrate significance (p = .083; see Table 4). This result 
is the same as in the model considering GPA at graduation with 
separate housing and Primetime variables. The ACT composite 
and high school four-point GPA are positively correlated (p < .001) 
with GPA at graduation, and first-generation status and participa- 
tion in one of the undergraduate colleges are negatively correlated 


(p <.01). 


TABLE 1. LINEAR REGRESSION MODEL OF THE IMPACT OF INDEPENDENT AND 
CoNTROL VARIABLES ON First-YEAR CoLLece GPA 
(Primetime, Honors Housinc) 
Variable 
(Constant) 
First Generation { -6.193* 
Sex : i i 4,019* 
College 
College B 1.350 
College C -7,961* 


l 
College D : 094 
College E 1.553 
ACT Composite 13,.700* 
High School 4-Point GPA : 12.067* 
Primetime : : : 1.545 


Honors Housing 2,656* 


*p<.05 

Notes: N = 3,259, F = 59.36, p < .05, R? = .155. The coefficient of determination indicates that the 
model explains 15.5 percent of the variation in first-year college GPA (R? = .155). This is only a 0.4 
percent improvement in explained variance over the model with all variables except honors housing 
and Primetime (R? = .151; results available from the authors upon request). 
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We evaluated four models using logistic regression: the effects 
of honors housing and Primetime as two binary variables on each 
of the binary variables indicating completion of honors coursework 
and graduation with University Honors and the effects of a single 
binary variable indicating participation in both first-year experi- 
ences on the same two dependent variables. Tables 5 and 6 present 
results for completion of 12 hours of honors coursework. The 
model including the two separate housing and Primetime variables 
(Table 5) did slightly improve fitness over a model excluding these 
variables. Both honors housing and participation in Primetime 
demonstrate a positive and significant (p < .05) correlation with 
completion of the coursework requirement. Primetime and honors 
housing produced the second- and third-highest effects on likeli- 
hood of completing honors coursework after high school GPA. 

A model including the single binary variable indicating partici- 
pation in both Primetime and honors housing and excluding the 
two separate variables for these experiences (see Table 6) produced 


TABLE 2. LINEAR REGRESSION MODEL OF THE IMPACT OF INDEPENDENT AND 
ConTROL VARIABLES ON First-YEAR CoLLece GPA 
(BotH Primetime AND Honors Housinc) 
Variable 
(Constant) 
First Generation | -6.228* 
Sex i { { 3.970* 
College 
College B 1311 
College C -8.004* 


l 
College D 168 
College E 1.565 
ACT Composite 13,847* 
High School 4-Point GPA 12.097* 


Both Primetime & Honors Housing ! ! ! 3,712* 


*p $05 
Notes: N = 3,259, F = 66.26, p < .05, R’ = .155. 
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a slightly less improved model, but the odds ratios (i.e., the prob- 
ability that the two variables are related) suggest a stronger increase 
in the likelihood of completing honors coursework. Specifically, 
students participating in both experiences were 2.32 times more 
likely to complete honors coursework compared to 1.79 times more 
likely for Primetime alone and 1.65 times more likely for honors 
housing alone. 

Tables 7 and 8 present results for the binary dependent variable 
indicating graduation with University Honors. The model includ- 
ing the two separate housing and Primetime variables (Table 7) 
again just slightly improved fitness over a model excluding these 
variables. Only honors housing demonstrated a significant (p < .05) 
correlation with University Honors graduation: students who lived 
in honors housing were 1.48 times more likely to graduate with 


TasBlLe 3. LINEAR REGRESSION MODEL OF THE IMPACT OF INDEPENDENT AND 
ConTROL VARIABLES ON GPA At GRADUATION 
(Primetime, Honors Housinc) 


(Constant) 
First Generation 
Sex 
College 
College B 


College C 
College E 
ACT Composite 
High School 4-Point GPA 


Primetime 


Honors Housing 


*p $05 

Notes: N = 578, F = 6.64, p < .05, R? = 095. The dummy variable for College D is not present in 
this model because there were no undergraduate degrees awarded through College D within this 
population. Assessing Primetime and honors housing separately indicates only a 0.3 percent increase 
in the explanation of variance in GPA at graduation (from an R? of .092 without housing and the 
Primetime variables to an R? of .095 when included). 
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University Honors. A model including the single binary variable 
indicating participation in both Primetime and honors housing 
and excluding the two separate variables for these experiences (see 
Table 8) again produced a slightly less improved model, and the 
odds ratios suggest a slightly stronger increase in the likelihood of 
completing honors coursework (1.56 times more likely, p < .05). 
Results from the qualitative survey feedback shed additional 
light on these findings regarding Primetime participation. The 
completion rate was very high, averaging 80 percent across the 
years of administration, and students were very positive in their 
responses. For example, about 80 percent of respondents either 
strongly agreed (42%) or agreed (38%) that their Honors Prime- 
time experience made them feel more confident about beginning 
their first semester of college, and 92 percent rated their overall Pri- 
metime experience as either excellent (38%) or good (54%). 


Tape 4. LINEAR ReGRession MobEL OF THE IMPACT OF INDEPENDENT AND 
ContTROL VARIABLES ON GPA at GRADUATION 
(Bot Primetime AND Honors Housinc) 
Variable 
(Constant) 
First Generation 
Sex 
College 
College B 
College C 
College E 
ACT Composite 
High School 4-Point GPA 


Both Primetime & Honors Housing 


*ps.05 

Notes: N = 578, F = 7.63, p < .05, R? = .097. The dummy variable for College D is not present in 
this model because there were no undergraduate degrees awarded through College D within this 
population. A regression model using the binary variable indicating participation in both Primetime 
and honors housing demonstrates only a 0.5 percent higher explanation of variance (R? = .097). 
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DISCUSSION 


The results of our preliminary study of entering honors students 
and comparable honors peers strongly suggested that the honors 
experience has a positive effect on student success as measured by 
GPA at the end of the first year and GPA at the time of graduation. 
Because the comparison group of honors peers was so similar to the 
group of honors students, it appears likely that specific components 
of the honors experience have had a positive impact on students’ aca- 
demic success. The results of the preliminary study prompted us to 
conduct the research presented in the main study, with its goal being 
to identify specific components of the honors program experience 
that contribute to an increase in honors students’ academic success. 

The main study considered two specific honors experiences, 
both of which occur during the first year: honors housing and a 


TaBLe 5. Loaistic REGRESSION MODEL OF THE IMPACT OF INDEPENDENT 

AND ConTROL VARIABLES ON Honors Coursework CompPLeTION 
(Primetime, Honors Housinc) 

Variable Odds Ratio 

(Constant) 

First Generation 

Sex 

College 

College B 


College C 
College E 
ACT Composite 
High School 4-Point GPA 


Primetime 


Honors Housing 
*ps<.05 
Notes: N = 2,610, p < .05, Nagelkerke R? = 139. The dummy variable for College D is not present in the 
cases used for this model. The two separate housing and Primetime variables shown in this table did 
slightly improve fitness over a model excluding these variables (the Nagelkerke R? increased from .093 
to 139, and classification improved from 61.3% to 63.2%). 
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pre-semester, credit-bearing class titled Honors Primetime. The 
data indicate that honors housing and Honors Primetime com- 
bined have a significant impact on the academic success of students 
in three of the four measures of student success we analyzed, as does 
living in honors housing as a separate variable. Although Prime- 
time participation only showed effects in coursework completion, 
a magnified effect on the success measures occurred when students 
participated in both opportunities. The study did not control for 
any selection bias: that is, students who choose honors housing and 
Primetime are likely not equivalent to students who do not. Con- 
trolling for a selection bias would necessitate a random assignment 
of honors students to honors residence halls, which was not pos- 
sible for this study. Yet, even without the control for a selection bias, 
the close nature of the comparison groups—honors students who 
chose to participate in honors housing and Primetime as compared 
to honors student who did not—suggests that these two first-year 
honors experiences added value to the students’ undergraduate 
experience. 


TaBLe 6. Loaistic REGRESSION MODEL OF THE IMPACT OF INDEPENDENT AND 
Controt VARIABLES ON Honors Coursework ComPLeTION 
(Bot Primetime AND Honors Housinc) 
Variable Odds Ratio 
(Constant) 
First Generation 
Sex 
College 
College B 
College C 
College E 
ACT Composite 
High School 4-Point GPA 


Both Primetime & Honors Housing 
*p $05 
Note: N = 2,610, p < .05, Nagelkerke R? = 125. The dummy variable for College D is not present in the 
cases used for this model. 
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Certainly, other factors, such as high school GPA and ACT/SAT 
scores, also affect academic success to varying degrees. For exam- 
ple, Campbell and Fuqua (2008) attempted to identify factors that 
were potential predictors of completion in their honors program, 
and they found that first-year GPA was most predictive, followed 
by honors housing. For that reason, this study also considered 
other factors that inform student success, such as high school GPA 
and ACT/SAT scores. The data in the study indicated that honors 
housing and Primetime still had a significant influence on student 
success, although other attributes, as would be expected, had a 
stronger influence. Such specific data on what enables student suc- 
cess allow honors programs to tailor the honors experience so that 
it better enables and benefits their particular student populations. 

Regarding the potential for added value of honors housing, 
prior research shows a positive effect of residence halls and in 


TaBLe 7. Loaistic REGRESSION MODEL OF THE IMPACT OF INDEPENDENT AND 

CONTROL VARIABLES ON GRADUATING WITH University Honors 
(Primetime, Honors Housinc) 

Variable ORE 

(Constant) 

First Generation 

Sex 

College 

College B 


College C 
College E 
ACT Composite 
High School 4-Point GPA 


Primetime 


Honors Housing 
*ps.05 
Notes: N = 578, p < .05, Nagelkerke R? = .123. The dummy variable for College D is not present in the 
cases used for this model. Including the two separate housing and Primetime variables, as shown in 
this table, again just slightly improved fitness over a model excluding these variables (the Nagelkerke 
R? increased from .109 to .123, and classification improved from 69.7% to 71.1%). 
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particular living-learning communities on student academic suc- 
cess (see Rinn 2004 for a review of the literature). Our findings 
suggest that honors housing in particular has an effect on student 
academic success above and beyond that of non-honors housing. 
As for Honors Primetime, its effect on student success was 
nearly equal to that of honors housing, and it was both popular 
and highly praised by its participants in the survey comments. This 
result contrasts in part to the findings of Perrine and Spain (2008), 
who saw that although participants in their optional six-day, pre- 
semester orientation program indicated via survey results that the 
experience helped with their academic and social adjustment to 
college, the data suggested that the orientation program had little 
effect on retention, credits earned, and GPA. Worth noting, how- 
ever, is that their orientation program was neither academic nor 
credit-bearing, as is Honors Primetime at the University of Iowa. 


TaBLe 8. Loaistic REGRESSION MODEL OF THE IMPACT OF INDEPENDENT AND 
CONTROL VARIABLES ON GRADUATING WITH University Honors 
(Both Primetime AND Honors Housinc) 
Variable Odds Ratio 
(Constant) 
First Generation 
Sex 
College 
College B 
College C 
College E 
ACT Composite 
High School 4-Point GPA 


Both Primetime & Honors Housing 
*p $05 
Notes: N = 578, p < .05, Nagelkerke R? = .118. The dummy variable for College D is not present in 
the cases used for this model. Including the single binary variable indicating participation in both 
Primetime and honors housing while excluding the two separate variables for these experiences, as 
shown in this table, did not improve the model (Nagelkerke R? = .118; classification = 70.6%). 
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As Perrine and Spain (2008) note, freshman orientation pro- 
grams in general are based on Vincent Tintos (1975) widely 
accepted concept of retention: that students’ feelings of connected- 
ness or their social integration into the campus community increase 
their commitment to the institution, and they are more likely to 
graduate. Yet, as Perrine and Spain (2008) again note, the evidence 
that orientation programs actually increase retention “is scarce 
and methodologically flawed,’ and those studies that are meth- 
odologically sound have shown mixed results (p. 156). It could be 
that orientation-like experiences benefit students in ways that are 
not normally tracked, such as their effect on alleviating the anxi- 
ety associated with transitioning into the university. Additionally, 
judging from comments generated by Honors Primetime, students 
are forming meaningful and lasting social relationships during 
the four-day experience. They are also getting to know faculty, the 
honors staff, and the campus better than their honors peers who 
do not take Primetime. Such benefits may not always show them- 
selves through GPAs, engagement in the program, and persistence, 
and yet they may well be valuable to students in other ways such 
as mental health. Finally, worth noting is the value experiences 
such as Primetime provide an honors program: a way to engage 
university faculty with honors students and the honors program as 
one of the twenty or so instructors of Primetime workshops. The 
results suggest that unique programmatic experiences that hon- 
ors programs often offer to honors students add value in terms of 
improved student outcomes, and thus the findings lend support for 
honors programs considering whether to offer pre-semester aca- 
demic experiences. 


IMPACT OF THE STUDY 


Verifying through data the positive impact of honors housing 
and Primetime has influenced our program in a number of ways. 
First, we now promote these opportunities much more strongly to 
our entering students. While we have not made them requirements, 
we actively highlight them to our entering students and their fami- 
lies during campus visits and, for Primetime, during summer 
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orientation for new students. Our professional staff speak about the 
benefits of honors housing and Primetime, but our honors program 
student ambassadors—those who have had these specific honors 
experiences—are our most effective advocates for them. Their per- 
sonal perspectives on living in Honors House or participating in 
Primetime are most beneficial in getting more of our entering class 
to select these options. 

The second benefit of a demonstrable positive impact of honors 
housing and Primetime has been in gaining institutional support. 
Recently, a university committee reviewed living-learning commu- 
nities on our campus. Sharing the results of our research on the 
positive effect of living in Honors House on student success clearly 
influenced the recommendations that the committee made. We 
have also been able to continue and grow Primetime with fund- 
ing from the Office of the Provost because of the demonstrated 
impact on student success and retention in honors. At a time when 
resource allocation strongly benefits from or even depends upon 
documented program effectiveness, the results of our study on the 
impact of Honors House and Primetime on our students’ success 
have greatly benefited the program and our students. 
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NOTE 


1. Twelve hours of honors coursework constitute half of the 
required honors curriculum and must be completed by the 
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student's fourth semester in the program. It is essentially a half- 
way check-in point. 


Address correspondence to Art L. Spisak at 
Art-Spisak@uiowa.edu. 
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na recent essay, M. Roy Wilson (2015), President of Wayne State 

University, and Jerry Herron, Dean of the Honors College, dis- 
cuss the value added of honors programs in terms that should be 
familiar to numerous constituencies associated with honors educa- 
tion. Wilson and Herron write about honors education largely in 
terms of the experiences it provides students: 


the [honors] college is not tied to any particular aca- 
demic discipline; instead, it represents the virtues of a 
liberal education that reaches across departments, schools, 
and colleges. For our students, the aim is to integrate 
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the specialized—and essential—knowledge of the disci- 
plines into a broader understanding of themselves, our 
community, and the world. With understanding comes 
engagement. The honors experience at Wayne State is based 
on four pillars—community, service, research, career— 
which define the curricular and co-curricular elements of 
our program and also highlight the distinctive strengths 
of this university, at the same time making real the value 
added, high-impact practices that define the very best of 
undergraduate education. (pp. 172-73) 


In this important conceptualization, honors education is about 
academic accomplishment but not in an isolated, discipline-spe- 
cific sense. Rather, the value added of honors hinges on something 
broader and perhaps less easily defined. Honors students may major 
in physics, engineering, or literature, but they are also immersed 
in an environment that forces them beyond these interests. Hon- 
ors education is about challenging students to focus on personal 
growth both as citizens and scholars, and in a poignant essay, Jac- 
queline P. Kelleher (2005) describes her experience as an honors 
student: 


It was about connectivity—to each other, to our families, 
to our community, and to global society. It was about not 
being afraid to try new things or work with new ideas; it 
was about digging deeper into a concept or message even 
when it was uncomfortable or downright impossible to 
understand. It was about admitting our failures and rec- 
ognizing our humanity. It was, and is still today. It served 
as the building blocks for the new path I carved out for 
myself. It was the scaffold I needed to discover who I was as 
a unique contributor to this world and what talents I could 
bring to the table of life. (p. 57) 


University administrators, honors deans, and scholars are also 
increasingly examining the concept of value added and honors in 
more concrete terms. As college costs increase and state support 
declines, administrators have become more dependent on tuition 
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dollars—driven by the recruitment and retention of students—to 
help drive university budgets. Recent work by Mitchell, Leachman, 
and Masterson (2017) highlights the extent of these changes and 
resulting pressures. Specifically, the authors find that state spending 
for public colleges and universities is still significantly below levels 
prior to the Great Recession of 2008 (Mitchell et al. 2017:2). At the 
same time, tuition at four-year public universities has increased an 
average of 35 percent since 2008 (Mitchell et al. 2017:3). Recession- 
related budget cuts contributed extensively to increased costs of 
public higher education, and these funds have generally not been 
replaced post-recession (Mitchell et al. 2017:5). One result is a cost 
shift from states to students; another is an increased concern among 
university administrators about recruiting and retaining students 
as a mechanism to mitigate budget concerns. 

Along with declines in state spending comes increased pressure 
to demonstrate what the university is getting in return for its invest- 
ment in a relatively small group of honors students. The challenge 
here is less on what honors programs provide in terms of a mean- 
ingful experience and personal and academic growth for students, 
but, rather, on how well they contribute to the parallel university 
goals of recruitment and retention. 

Of course, the broad goals of honors education and university 
recruitment and retention are not necessarily mutually exclusive. 
As West (2014) points out, an honors college can provide value to a 
university by recruiting high-achieving students who likely would 
not enroll were it not for the honors environment. In addition, 
the nature of the honors setting, highlighted by quality instruc- 
tion, small class sizes, closer relationships with professors, and a 
sense of intellectual community, may foster increased retention 
among honors students (West 2014). Taken together, then, we see 
the potential value added of honors education in terms of not just 
the opportunities for intellectual and personal growth that honors 
programs accrue to honors students and faculty, but also as an addi- 
tional resource for university administrators as they wrestle with 
the increasingly complex financial realities of higher education. 
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THE ROLE OF THE HONORS COLLEGE IN RECRUITMENT, 
RETENTION, AND STUDENT SUCCESS 


In this paper, we address the issue of value added in honors edu- 
cation by examining recruitment, retention, and student success at 
the University of Mississippi (UM) and its Sally McDonnell Barks- 
dale Honors College (SMBHC). We utilize survey data of honors 
students as well as a matching analysis comparing honors students 
and the overall student body across a variety of subgroupings and 
outcomes. Our goal is to add to the growing body of research that 
more clearly and rigorously delineates the possible impact and 
value of honors programs at state public universities. 

Using a variety of methodological techniques, previous stud- 
ies generally show a positive and significant influence for honors 
college participation on student retention and academic success, 
especially in the initial years on campus. Cosgrove (2004), for 
example, compares three sets of students: honors students who 
completed all program requirements, honors students who did not 
complete all requirements, and non-honors students with similar 
incoming academic credentials. Cosgrove (2004) finds that hon- 
ors program “completers” perform better than their partial honors 
and comparable non-honors counterparts in terms of college GPA, 
graduation rates, and length of time to graduation. Shushok (2006) 
finds that honors students at the University of Maryland, College 
Park had higher first-year GPAs and retention rates at the end of 
the first year, though statistically significant differences in rates of 
retention disappeared by the fourth year. Slavin, Coladarci, and 
Pratt (2008), used logistic regression to examine retention and 
graduation rates at the University of Maine and found similar 
results of significant one-year effects that dissipated by the fourth 
year. In addition, Keller and Lacy (2013) found that participation 
in the honors program at Colorado State University “was associ- 
ated with meaningful increases in the proportion of these students 
who returned for their second year at the university and in the pro- 
portion of these students who graduated within a four-, five-, or 
six-year period” (p. 83). 
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Setting 


The University of Mississippi is the state’s flagship university, 
as well as its largest, enrolling 23,780 students across its main, 
regional, professional, and Medical School campuses.’ The Univer- 
sity of Mississippi is also ranked among the nation’s fastest-growing 
institutions, with the student body at the main and regional cam- 
puses growing approximately 60 percent during the past decade. 
Overall enrollment on these campuses is currently at 20,890 (86.7% 
undergraduate). Undergraduates entered fall 2017 with an aver- 
age ACT score of 25 and high school GPA of 3.59. Undergraduate 
minority enrollment is 22.1 percent (12.5% African American). 
The undergraduate student body is 54.3 percent Mississippi resi- 
dents and 54.8 percent female.’ 

Endowed in 1997, the honors college has also exhibited signifi- 
cant growth during the past twenty years, and it is currently home 
to over 1,500 undergraduates, including an average entering class 
of approximately 400 first-year students. These students represent 
38 states and 11 countries, and they are engaged in academic pur- 
suits across the broad spectrum of disciplines at the University of 
Mississippi. Fifty-four percent of honors students are Mississippi 
residents, 63 percent are female, and 12 percent are minorities 
(3.6% African American). The average ACT for the most recent 
first-year class is 31, along with an average high school GPA of 3.99. 

The honors environment is one where students are encour- 
aged to merge intellectual rigor with community action, fostered 
initially by a two-semester first-year seminar sequence focusing 
on broad intellectual themes, difficult social issues, contemporary 
challenges, and self-discovery. In addition, honors students have 
access to a first-year student living-learning community residence, 
advising (including a dedicated advisor for national scholarships), 
travel fellowships, and experiential learning. Class sizes are capped 
at 15 students and are expected to be seminar-based and contain 
significant writing and critical thinking components. Part of the 
honors experience is engaging in the “Community Action Chal- 
lenge,’ which encourages students to become agents of change in 
their own community. Finally, honors students must accumulate 
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a minimum of 30 credit hours in honors courses, maintain a 3.5 
GPA, and successfully write and defend an honors thesis to gradu- 
ate with honors. 


Data 


We examined the value added of the honors college using two 
data sources. In spring 2016, we undertook a survey of currently 
enrolled (or recently graduated) students in the honors college. 
After receiving IRB approval, we contacted 1,091 students from the 
four most recent academic classes (entering in the fall semesters of 
2012-2015) to distribute surveys using the online survey software 
Qualtrics.? Of the 1,091 students emailed, we received 521 com- 
pleted surveys for a response rate of 47.8 percent. Survey responses 
were broken down by academic class (year of entering fall semester) 
as follows: 15.2 percent (n = 79) were from the class of 2012; 25.9 
percent were from 2013 (n = 135); 24.2 percent were from 2014 (n 
= 126); and 34.7 percent (n = 181) were from 2015. Students were 
asked a battery of questions allowing us to examine factors related 
to student recruitment, satisfaction, and success at the honors col- 
lege. Of particular interest for our purposes here were perceptions 
of the prestige of the honors college (both individually and relative 
to the university as a whole) and whether the student would have 
attended the university were it not for the honors college. 

In addition, we also made use of administrative data from 
honors and university student data systems. To determine factors 
that influence progress in the program, we examined the path of 
students through both the honors college and the university from 
2012-2015. For honors students, we evaluated the entering classes 
from each of the last four years for a sample size of 1,503 students. 
We then matched these student attributes with similar data for the 
overall student body. We discuss the process of data matching in 
greater detail below. Using information collected in the applica- 
tions and other updated information as the students completed 
each semester, we analyzed factors that may have a significant influ- 
ence on their likelihood of staying active in the honors college as of 
the end of the spring semester in 2016. 
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FINDINGS 
Value Added and Recruitment 


Our principal interest in examining honors student survey data 
was to determine if the honors college does provide added value to 
the university via recruitment of higher achieving students. Follow- 
ing recent work by Nichols and Chang (2013), we asked students 
to respond to a battery of possible influences on their initial deci- 
sion to enroll in the honors college. Similar to the case in Nichols 
and Chang (2013), respondents overwhelmingly cited aspects of 
the honors college that distinguish it from the regular university 
environment as prominent factors in their decision to attend the 
university and its honors college. The availability of small class sizes 
was cited as most influential in the decision to enroll, with 77.6 
percent of respondents noting this factor as either “very influential” 
or “extremely influential” in their decision to attend. Opportunities 
for deeper learning (63.6%) and the possibility of research, travel, 
and leadership opportunities (58.8%) are also strong influences, as 
was the opportunity to make meaningful connections with faculty 
(52.8%). 

Of additional interest were differences in student evaluations 
of the perceived prestige of both the honors college and the uni- 
versity as influences on the decision to attend the university. Table 
1 presents the results of these comparisons. From this initial look 
at the data, we see that the honors college provides added value as 
a recruitment tool for the university. Almost 60 percent of respon- 
dents cited the prestige of the honors college as very or extremely 
influential on their decision to attend. Students were over twice as 
likely to report that the reputation of the honors college was very 
or extremely influential as they were to report that the reputation 
of the university was very or extremely influential in their decision. 
We find important differences at the other end of the evaluative 
spectrum as well. Whereas just 3.6 percent of honors students indi- 
cate that the prestige of the honors college was not influential for 
their decision to enroll, one quarter (25.8%) viewed the prestige of 
the university as not influential for their decision. A chi-square test* 


185 


BROWN, WINBURN, AND SULLIVAN-GONZALEZ 


comparing responses to the perceived prestige of the honors college 
and the university confirms that honors students are significantly 
more likely to credit the honors college with their decision to enroll. 

We explored the potential recruitment draw of the honors col- 
lege further by asking a direct question about the impact of honors 
and the decision to attend the university: 


One final question: would you have attended the University 
of Mississippi if you had not been accepted to The Sally 
McDonnell Barksdale Honors College? 


Table 2 shows initial results, along with comparison by residence 
(in-state vs. out-of-state) and ACT score (31 and below vs. 32 and 
above).° Beginning with the entire set of respondents, we see that 
over half (51.7%) indicated it is likely (perhaps or definitely) that 
they would have attended the university even if they had not been 
accepted into the honors college. Importantly, however, well over 
one third (37.3 percent) responded that it was “unlikely” or that 
they would have “definitely not” attended if not for the invitation 


Tasle 1. REASONS FoR DECISION TO ENROLL: COMPARING PERCEIVED 
PresTIGE OF Honors COLLEGE AND UNIVERSITY (PERCENT) 


Response Prestige of Honors College —_ Prestige of University 
Extremely Influential 


Very Influential 


Influential 


Somewhat Influential 


Not Influential 


Notes: Numbers in parentheses are frequencies. A X* test for this contingency tables indicates that the 
association between student response and perceived prestige is significant (X? = 152.88, df= 16, p<.001). 
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to join the honors college. In other words, based on these survey 
responses, almost 150 students from an incoming class of 400 hon- 
ors students probably would not be attending UM were it not for 
the honors college. 

While a substantial percentage of students self-reported that 
they would probably not have attended UM without the honors col- 
lege, this figure varies by both residence and ACT score. A much 
higher percentage of non-residents indicate they would have been 
unlikely to attend (49.4% compared to 27.5% for Mississippi resi- 
dents). Similarly, students with the highest ACT scores are more 
inclined to say that they would not have attended the university 
without acceptance into the honors college (47.2%). Chi-square 
tests indicate that both of these associations are significant (Table 2). 

Breaking down these data by both residency and ACT score 
reveals further insights into the impact of the honors college on 
UM enrollment, particularly among Mississippi's highest achievers 
on the ACT (Table 3). The students most likely to respond that they 
would have attended UM even without admission into the honors 
college are those who reside in-state and who have an ACT score at 
or below the 31 average for the honors college. Of these students, 
almost three-quarters (72.3%) said that it was likely that they would 


TaBLe 2. REPORTED LIKELIHOOD OF ATTENDING UM IF Not For 
ACCEPTANCE AT Honors COLLEGE (PERCENT) 
Mississippi  Non- 
Response Overall Resident Resident ACT<32 ACT2>32 
Perhaps or Definitely 51.7 60.7 40.5 62.4 
(196) (128) (68) (136) 
Neither 111 11.9 10.1 7.8 


(42) (25) (17) (17) 
Unlikely or Definitely Not. 37.2 27.5 49.4 29.8 
(141) (58) (83) (65) 


Notes: Numbers in parentheses are frequencies. A X? test indicates that the association between student 
response and residency is significant (X? = 19.7, df= 2, p $ 001). A X* test indicates that the association 
between student response and ACT score is significant (X? = 23.82, df =2, p< .001). 
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have attended UM even if they had not been admitted to the honors 
college. In other words, these are students with a strong likelihood 
of enrolling at the university, regardless of their acceptance into the 
honors college. 

When looking at their counterparts with the highest ACT 
scores (32 and above), however, we find that the honors college 
does indeed add significant value to the appeal of the university 
in terms of recruiting high-quality Mississippi students. Compared 
to the almost three-quarters of students in the 31 and below ACT 
category, only 42 percent of Mississippi residents in the higher ACT 
group indicated that they would likely have attended UM without 
acceptance into the honors college. On the other end of the spec- 
trum, Mississippi residents in the 32+ ACT group were nearly twice 
as likely to respond that it was “unlikely” or that they would “defi- 
nitely not” have attended the university if not for the honors college 
admission (20.1% vs. 39.5%). A chi-square test reveals this associa- 
tion between the likelihood of a Mississippi resident attending UM 


TasLe 3. REPORTED LIKELIHOOD OF ATTENDING UM Ir Not For 
ACCEPTANCE AT Honors CoLtece, BY RESIDENCY AND ACT 
(PERCENT) 


In-State Out-of-State 
RES TOs Overall ACT<32 ACT232 ACT<32 ACT2>32 
Perhaps or Definitely ol 42.0 47.3 32.5 
(196) (34) (42) (26) 


Neither 111 18.5 8.0 125 
(42) (15) (7) (10) 
Unlikely or Definitely Not. 37.2 39.5 44,3 55.0 
(141) (32) (39) (44) 


Notes: Numbers in parentheses are frequencies. A X? test indicates that an association between student 
response and ACT score for Mississippi residents is significant (X* = 19.41, df= 2, p < 001). AX? test 
indicates that an association between student response and ACT score for Mississippi non-residents 
is not significant (X* = 4.22, df = 2, p < .121). AX? test indicates that an association between student 
response and residency for those in the < 32 ACT group is significant (X* = 15.5, df= 2, p < 001). A X? 
test indicates than an association between student response and residency for those in the ACT > 32 
group is not significant (X? = 3.95, df= 2, p < .138). 
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with honors acceptance and ACT score is significant (y” = 19.41, 
df = 2, p < .001). In terms of helping to stem the brain drain and 
bringing the state’s best students to UM, the honors college offers 
significant value added. 

Similar results are apparent in other comparisons across the 
residency and ACT groups. Not surprisingly, the impact of honors 
admission appears strongest for non-residents in the 32+ ACT cat- 
egory because they are students with both the fewest ties to the state 
and the most academic options. Of this group, 55 percent indicate 
that they likely would not have attended the university without the 
offer of admission from the honors college. In contrast, only 32.5 
percent indicated it was likely that they would have attended with- 
out the honors admission. 

As noted in Table 3, chi-square tests indicate that these asso- 
ciations between residency-test categories and student response are 
significant with two important exceptions. First, for nonresidents, 
those students who indicate the least likelihood of attending UM 
without the honors college, ACT score is not significantly associ- 
ated with their response on the likelihood of attending without 
honors admission. Without the honors college, high-achieving non- 
residents would be less likely to attend UM regardless of their ACT 
score. In addition, we find no significant association between hon- 
ors acceptance and the likelihood of attending UM when comparing 
the highest-achieving residents and non-residents: those in the 32+ 
ACT category. This survey reveals that Mississippi's highest-achiev- 
ing residents report a similar impact of honors admission on their 
decision to attend UM as their counterparts from other states. 

These data inform some concluding observations regarding the 
honors college and recruitment to UM. With regard to Mississippi 
residents at or below the honors college average ACT score, there 
is modest value added in terms of honors recruitment. These stu- 
dents are highly likely to attend the university regardless of honors. 
This result is not terribly surprising because this group most closely 
resembles the student body writ large. Yet when we examine the 
impact of honors college admission across a range of residency and 
ACT comparisons, we can isolate those students whom we can rea- 
sonably expect to have increased options, both in terms of schools 
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and scholarship opportunities. Here the value added component 
of honors status becomes obvious. Our data reveal the honors col- 
lege to be a significant component in the decisions of Mississippi's 
highest-achieving students to attend the university. One signifi- 
cant additional consequence is that attracting outstanding students 
from other states has a strong impact on the diversity of the univer- 
sity student body. 


Value Added and Retention, Completion, and 
Academic Success 


We also examine whether the honors college serves a func- 
tion in retaining students. Year-to-year, even semester-to-semester, 
retention is an important consideration for universities today, and 
any honors program that increases retention provides added value. 
While proponents of honors programs often presume higher aca- 
demic performance and retention rates among honors students, 
determining the role that participation in an honors program con- 
tributes to this success can be problematic. For example, given the 
nature of the university populations, honors students often enter 
the academy with a greater likelihood of success in the first place. 

Because of the role of the honors college in recruiting students 
to campus, we examine whether students who start in the honors 
college are more likely to stay at the university, have higher first- 
year GPAs, and are more likely to graduate than those who did not 
start in the honors college. In the analyses in Tables 4-8, we focus 
on retention, academic performance, and graduation rates among 
enrolling students at the University of Mississippi between fall 
2012 and fall 2015. The dependent variables are whether students 
returned for the fall semester of their second (2012-2015 cohorts 
included in the analysis), third (2012-2015), and fourth years 
(2012-2014). For the 2012 cohort, we also examined whether a stu- 
dent graduated in four years. Finally, we evaluated an additional 
measure of student success: GPA after the first year for all years in 
the data. We classified honors participation as initial enrollment 
in the program and if the student remained in good standing in 
the honors college thereafter (Keller and Lacy 2013). Additionally, 
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we dropped all non-honors students with an ACT lower than 21 
from the analysis because the honors college did not have any 
participants with an ACT score lower than 21. We dropped some 
students who had missing data on any of the variables included in 
the analysis; otherwise, each student appears as one observation in 
the analysis. 

Descriptively, Table 4 shows the performance of honors and 
non-honors students on the dependent variables. We find that hon- 
ors students outperform their non-honors peers across all measures 
with retention rates between 10 and 22 percentage points higher, a 
graduation rate that is 29 percentage points higher, and a GPA that 
is .71 higher. These differences suggest initial support for the value 
added of honors education in retention and student success. 

These descriptive results do not, however, control for any back- 
ground factors that may be driving the results beyond participation 
in the honors college. To account for this factor, we include various 
background demographics and academic performance measures as 
controls. Table 5 shows the control variables by honors status. We 
controlled for demographics including being a minority (simplified 
to whites vs. nonwhites) and gender. To capture academic perfor- 
mance, we use both ACT scores and high school GPA. We also 
controlled for student origin with three groups: in-state students 
from Mississippi and two out-of-state categories: South® and non- 
South. Finally, we included beginning academic year as a control to 
account for any unmeasured differences across cohorts. 


Table 4. Descriptive OuTcoMES 


Measure Honors Non-honors(ACT>21) Difference 
Return Year 2 (2012-2015) 96.67 
Return Year 3 (2012-2015) 94.08 


Return Year 4 (2012-2014)* 92.29 
Graduated in Four Years (2012)° 79.51 
GPA After Year 1 (2012-2015) 3.62 


* The n is 1,103 for honors and 6,934 for non-honors (ACT > 21). 
>The n is 371 for honors and 2,115 for non-honors (ACT > 21), 
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To measure the effect of honors program participation while 
controlling for other important factors, we used matching analy- 
sis. This type of study fits nicely with the matching framework of 
isolating treatment effects, such as honors college participation, 
in observational studies. A matching analysis matches students in 
the honors college to those most similar to them not in the hon- 
ors program to provide a point of comparison for evaluating the 
effect of honors on academic performance and the likelihood of 
remaining in school and graduating. This method is similar to the 
one employed by Keller and Lacy (2013). We used a “nearest neigh- 
bor” approach in which the model selects the non-honors student 
or students who match the closest on the control variables with 
the model specifying exact matches on race, gender, and region. 
Specifically, the models use the Mahalanobis distance matching 
(MDM) procedure, which matches each treated unit to the near- 
est control unit within a specific distance. According to King et al. 
(2011), the MDM approach then removes treated units that do not 


Taste 5. Descriptive STATISTICS FOR CONTROL VARIABLES 


Measure Honors Non-Honors (ACT > 21) 
Minority 11.64 (175 


) 14.58 (1,428) 
Female 62.61 (941) 54.64 (5,353) 
High School GPA* 3.94 (.1375) 3.49 (.4382) 
ACT? 30.29 (2.413) 24.94 (3.003) 
Region 
Mississippi 56.09 (843) 37.03 (3,628) 
South 31.00 (466) 43.61 (4,272) 
Non-South 12.91 (194) 19.36 (1,897) 
Year 
2012 24.68 (371) 21.59 (2,115) 
2013 24.68 (371) 23.63 (2,315) 
2014 24.82 (373) 25.92 (2,539) 
(2,828) 


2015 25.82 (388) 28.87 (2,828 


Note: Numbers in parentheses are group frequencies except where noted. 
“Numbers in parentheses for this row are standard deviations. 
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have a match within the specified distance. As Dehejia and Wahba 
(2002) discuss, this method reduces bias. The MDM approach may 
lead to less precise estimates when a large number of treated units 
must be removed from the model because of a lack of a matching 
unit within the specified distance, while propensity score match- 
ing, acommon alternative, may produce more precise estimates but 
introduce more bias into the estimates (Dehejia and Wahba 2002). 
King and Nielsen's (2016) recent work also echoes the concerns 
of bias using propensity score matching. Given that relatively few 
treated units (honors students) are removed using the MDM pro- 
cedure, we proceeded with the nearest-neighbor approach rather 
than propensity score matching. As a check for robustness, we ran 
the models using propensity score matching: the results did not 
change substantively. We specifically used exact matches for minor- 
ity, gender, and regional status and adjusted for potential bias on 
the continuous variables high school GPA and ACT score.’ 

Table 6 presents the average treatment effect on the expected 
difference in potential outcomes among individuals who partici- 
pated in the honors college. This result is the effect of honors college 
membership on retention or performance while controlling for the 
other variables. Table 6 shows a significant and positive influence 
for honors college participation compared to what would have 
been expected had these students not participated in honors. Com- 
pared to their nearest-neighbor matches, honors college students 
were 2.73 percent more likely to return for year two, 5.03 percent 
more likely to return for year three, and 10.12 percent more likely 
to return for year four. They also were 8.36 percent more likely to 
graduate in four years although this effect was only marginally sta- 
tistically significant (p = .08). Finally, honors participation led to a 
statistically significant increase of GPA of .17 points after the first 
year. 

While these expected differences are smaller than the differ- 
ences observed in the descriptive analysis presented in Table 4, the 
results highlight the estimated unique effect of honors college par- 
ticipation, adjusting for the combined effects of all of the control 
variables (Keller and Lacy 2013). For example, participation in the 
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honors college accounts for 2.73 percentage points of the 10.95 per- 
centage point difference between honors and non-honors students 
in Table 4 and 10.12 percentage points of the 22.40 percentage 
point difference for returning in year four. The 2.73 percentage 
point difference is slightly higher than that reported for Colorado 
State students by Keller and Lacy (2013); the 10.95 percentage point 
observed difference is very similar to the unadjusted difference 
reported in that study. 

These results suggest a value added for students participating 
in the honors program. To test if honors participation has more 
of an effect on certain types of students in the program, we broke 
the sample into two groups: those with ACT scores above aver- 
age for honors students and those with ACT scores at or below the 
average. While we controlled for ACT in the model in Table 6, this 
additional analysis allowed us to focus specifically on the groups 
of interest discussed earlier in the paper. Table 7 shows the MDM 
results for those students with ACTs above the honors college aver- 
age. There are 502 honors students with an ACT greater than or 
equal to a 32 and 272 “nearest neighbor” non-honors students in 
the sample. For these high-achieving students, the treatment effect 
of the honors experience is not significant for retention in years two 
and three, nor is it for the likelihood of graduating in four years for 
the 2012 cohort. We do, however, find significant and substantively 
large effects for the retention in year four and for first-year GPA. 


Taste 6. Nearest NEIGHBOR MaTcHING RESULTS 
Outcome Expected Difference Robust SE 
Return Year 2 (%) LOO” 11,234 
Return Year 3 (%) 0144*** 11,234 


Return Year 4 (%) {nispeers 7,996 
Graduated in 4 Years (%) 0469 2,386 


First-Year GPA : 122 aaa 11,234 
*p $05, **ps.0l, **ps 001 
Notes: The expected difference is the average effect of honors experience vis-a-vis the comparable 
“nearest neighbor” students. In the Year 2, Year 3, and First-Year GPA models, 66 observations were 
dropped due to a lack of suitable matches. In the Year 4 model, 41 observations were dropped, and 
in the Graduated in 4 Years model, 100 observations were dropped due to a lack of suitable matches. 


194 


RECRUITMENT, RETENTION 


These results suggest that participating in the honors program offers 
some value on retention and academic performance beyond what 
we would normally expect for otherwise high-achieving students. 

Table 8 presents MDM results for those students with ACT 
scores at or below the honors college average. There are 1,001 hon- 
ors students and 9,525 “nearest neighbor” non-honors students in 
the sample with ACTs less than 32. The results show a similar pat- 
tern and substantive results comparable to those in Table 6, which 
is not surprising given most of the overall sample comes from this 
group. The one notable exception is the significant influence of 
honors participation on the likelihood of graduating in four years 
(cf. 10.71% [p < .05] to 8.36% [n.s.] in Table 6). 

Overall, the results in Tables 6-8 show similar patterns to those 
in previous studies on retention from across the country. Generally, 
students are more likely to be retained from year one to year two 
with significantly higher first-year GPAs when they participate in 
the honors program. The results also show some influence for reten- 
tion in years three and four, with a notable bump for high-achieving 
students in year-four retention. While honors participation did not 
significantly increase the likelihood of graduating in four years in 
the overall model (Table 6) or for high-achieving students (Table 
7), it was significant for those students with an ACT at or below the 
honors college average (Table 8). The matching method allows for 


TasLe 7. Nearest NEIGHBOR MATCHING RESULTS 
(StupenTs with ACT 2 32) 
Outcome Expected Difference Robust SE 
Return Year 2 (%) : 0178 
Return Year 3 (%) 0435 


Return Year 4 (%) .0505** 
Graduated in 4 Years (%) 1521 
First-Year GPA : .0647*** 


*p $05, **ps.0l, ps 001 

Notes: The expected difference is the average effect of honors experience vis-a-vis the comparable 
“nearest neighbor” students. In the Year 2, Year 3, and First-Year GPA models, 5 observations were 
dropped due to a lack of suitable matches. In the Year 4 model, 23 observations were dropped, and 
in the Graduated in 4 Years model, 15 observations were dropped due to a lack of suitable matches. 
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an isolation of the unique effect of honors participation by creating 
a comparable group of students who do not participate in honors 
across a range of other variables, such as demographic features 
or high school academic performance, in the student population. 
Analyses using this methodological approach show that honors 
participation does produce better retention and academic perfor- 
mance outcomes. 

Combining these results with those in the analysis of university 
student recruitment, we find that while students with an ACT at 
or below a 31 are very likely to attend the university regardless of 
honors participation, being a part of the honors program increases 
their likelihood of staying at the university and having greater ini- 
tial academic success. For those students with an ACT greater than 
the honors college average, being participants in honors is a major 
reason why they chose to attend the university. This participation, 
while not as strongly associated with retention, does provide a boost 
to their academic success and long-term retention at the univer- 
sity. The value of the honors college is in attracting both the state's 
top students and those from out of state. Additionally, for those 
students most likely to attend the university regardless of honors 
admission, the honors college experience increases their academic 
performance relative to comparable non-honors students. 


TasBLe 8. Nearest NEIGHBOR MATCHING RESULTS 
(StupenTs witH ACT < 32) 
Outcome Expected Difference Robust SE 
Return Year 2 (%) .0120** 10,465 
Return Year 3 (%) JOE 10,465 


Return Year 4 (%) .0207*** 7,421 
Graduated in 4 Years (%) .0498* 2,247 
First-Year GPA : 10285 ame 10,465 


*p $05, **ps.0l, ps 001 

Notes: The expected difference is the average effect of honors experience vis-a-vis the comparable 
“nearest neighbor” students. In the Year 2, Year 3, and First-Year GPA models, 61 observations were 
dropped due to a lack of suitable matches. In the Year 4 model, 116 observations were dropped, and 
in the Graduated in 4 Years model, 97 observations were dropped due to a lack of suitable matches. 
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DISCUSSION AND CONCLUSIONS 


The concept of value added in honors education can be exam- 
ined from multiple perspectives. From the point of view of the 
prospective or current honors student, the hope is that the hon- 
ors college experience seems qualitatively different, characterized 
by an environment where small classes with other intellectually 
gifted and curious students create opportunities for deeper learn- 
ing, connections with faculty, and personal growth that transcend 
the typical large-enrollment university model. 

For university administrators working within a context charac- 
terized by increasing resource constraints, value added may take on 
a different dimension altogether. While administrators recognize 
the intrinsic value of what honors education may provide students, 
bottom-line budget realities factor in here, and return on invest- 
ment becomes an increasing concern. 

Our analyses of honors education at the University of Mississippi 
illustrate that these perspectives on value added are more compat- 
ible than they are contradictory. In examining factors related to 
the recruitment of honors students, we find that the honors setting 
does attract students looking for an educational experience that is 
different from what they are likely to receive in a more traditional 
university environment. Here we see the draw of the honors college 
providing a richer educational experience to students. The result is 
that honors provides value added to the institution by recruiting 
students who indicate that they would otherwise not have attended 
the university. 

In addition, our survey of honors students reveals important 
differences in how students perceive the relative prestige of the hon- 
ors college versus the university, and these differences are salient to 
enrollment. The perceived prestige of the honors college represents 
an important inducement and provides significant value added in 
recruitment when compared to the perceptions of the prestige of 
the university overall. 

Finally, our survey of current honors students reveals that the 
honors college provides significant value added to the university 
by helping to recruit students who indicate that they would not 
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have attended the university were it not for the honors college. 
Most importantly, this recruitment impact draws Mississippi's 
highest-achieving high school students as measured by ACT to the 
university and helps diversify the student body by attracting aca- 
demically strong out-of-state students. 

With regard to retention of students, we see solid evidence of 
value added here as well. Our matching analysis comparing honors 
to non-honors students shows results consistent with other recent 
work in this area, with honors students showing statistically signifi- 
cant differences in first-year GPA as well as a higher likelihood of 
returning to the university in each subsequent year of their college 
careers. More focused comparisons lead to somewhat attenuated 
results, but the overall pattern is solid and indicates significant 
effects in terms of student success and retention. 

While we do not have survey data that allow us to tap into the 
mechanisms that are driving these retention results, we can specu- 
late. On the one hand, some aspect of a selection effect is likely 
involved here. Students who apply to the honors college may well 
possess traits that differentiate them from their academically simi- 
lar counterparts who do not apply, and these traits may be related 
to retention. At the same time, however, it is also likely that the 
honors environment that attracts these students in the first place is 
also successful in providing them with an academic experience that 
fosters the intellectual and personal growth that they seek and that 
the honors environment and experiences translate into increased 
academic success and retention. Combined with the impact on 
recruitment, these results for retention and student success show 
how value added for the broad goals of honors education and 
value added in an environment of constrained resources may go 


hand-in-hand. 
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NOTES 


1. Fall 2017 enrollment data. 


2. UM has a Carnegie classification of Doctoral University: Highest 
Research Activity. 


3. IRB protocol (number 16x-255) applies to all analyses. 


4, Chi-square is a common measure of association for testing rela- 
tionships between categorical variables. 


5. The two categories for ACT score break the distribution roughly 
in half around the average ACT score (31). 
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6. Students from the remaining 10 states of the former Confederacy. 


7. This strategy accounts for potential estimator inconsistency and 
bias when using two or more continuous variables in a nearest- 
neighbor model (Abadie and Imbens 2006; 2011). 


Address correspondence to Robert Brown at 
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A Propensity Score Analysis 
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Peas to Morgan and Badenhausen (2015), honors educa- 
tion began in the United States in 1921 when Frank Ayedelotte 
became president of Swarthmore College. At that time, Ayede- 
lotte initiated an interdisciplinary curriculum that stressed critical 
thinking and active learning. Almost a century later, the National 
Collegiate Honors Council (2013) defines honors education in 
terms true to Ayedelotte'’s original vision: 


Honors education is characterized by in-class and extra- 
curricular activities that are measurably broader, deeper, 
or more complex than comparable learning experiences 
... [and] honors experiences include a distinctive learner- 
directed environment and philosophy. (para. 2) 


Similar to four-year university honors programming, community 
colleges have likewise established honors programs to meet the 
academic needs of high-achieving students. Floyd and Holloway 
(2006) recall that community colleges introduced honors programs 
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in the 1950s and 1960s in the form of “accelerated courses offered 
to academically talented students who had expressed interest 
in specific areas of study ” (p. 43). In the 1980s, community col- 
leges broadened their enrollment focus from open enrollment and 
social equality to increased attention to academic excellence. Car- 
nicom (2011) reasons that increasing quality and academic rigor, 
especially with regard to transfer courses, led to an expansion of 
community college honors programming. The National Collegiate 
Honors Council (2017) lists 190 community college members, rep- 
resenting 20 percent of its membership. 

Armstrong and Jones (2015), Bullock and Fennell (2015), 
Burrage and Coleman (2015), and many other honors program pro- 
ponents contend that honors programs have the potential to make 
an important difference in postsecondary education. Although 
intriguing scholarship regarding community college honors edu- 
cation is developing, continued research, particularly with regard 
to community college honors programming, is a priority in order 
to answer the basic question of whether or not community college 
honors program participants emerge from the experience with out- 
comes superior to those of comparable non-honors students. 

Keller and Lacy (2013) point to the significance of current 
quantitative research on college-level honors programs that has 
employed propensity score techniques because this method bol- 
sters causal arguments by decreasing selection bias. Austin (2011) 
defines the propensity score as 


a balancing score: conditional on the propensity score, 
the distribution of measured baseline covariates is similar 
between treated and untreated subjects. Thus, in a set of 
subjects all of whom have the same propensity score, the 
distribution of observed baseline covariates will be the 
same between the treated and untreated subject. (p. 402) 


In the present study, the treatment cases are defined as honors pro- 
gram participants (honors students), those students who completed 
12 or more honors credit hours. The untreated cases, on the other 
hand, are defined as honors-eligible nonparticipants (non-honors 
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students). The propensity score analysis we employ is based on a 
more extensive number of confounding variables than has been 
characteristic of previous research. This study adjusts for 13 vari- 
ables, including ACT combined and sub-scores, high school GPA, 
socioeconomic status, first-generation college student status, and 
other characteristics also linked to academic success and honors 
participation. Following propensity score analysis, we conducted 
independent samples t-tests, which determine if a significant dif- 
ference exists between the averages of two unrelated groups, and 
Pearson chi-square analyses, which determine whether a statisti- 
cally significant difference exists between expected and observed 
rates between groups to estimate the unique effect of honors pro- 
gram participation. 


RATIONALE FOR COMMUNITY COLLEGE HONORS 


Bullock and Fennell (2015) note that community colleges have 
become a focus of national attention as leaders acknowledge the 
outstanding progress that community college students are making: 
“Thanks to the efforts of hard-working, dedicated faculty and for- 
ward-thinking college leaders, test scores, grades, and completion 
rates are making slow but steady progress while achievement gaps 
are diminishing” (p. 27). Nevertheless, Trucker (2014) cautions that 
two-year colleges remain relatively low in terms of conventional 
measures of retention and graduation: 


longitudinal studies that track student persistence each 
semester serve as the primary measurement of an institu- 
tion’s success or, as the findings are often received at many 
of the country’s community colleges, an institution’s failure. 
These studies take place at the institutional and state-wide 
levels as well as nationally through grant-based organi- 
zations such as Complete College America. . . . [T]hese 
studies consistently reveal low college-wide retention and 
graduation rates. (p. 69) 
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According to the Century Foundation (2013), community col- 
leges, which serve approximately 11 million students in the United 
States, are expected to educate the most at-risk students while 
expending minimal financial resources in institutions that are 
becoming more and more unconnected and dissimilar to four-year 
universities; thus, American higher education reflects the growing 
inequality in the larger society. To encourage racial and economic 
inclusiveness, the Century Foundation (2013) recommends that 
two-year colleges invest in innovative honors programming 
because honors programs attract high-achieving, economically 
disadvantaged students. Gee and Blemings (2015) further contend 
that community college honors programs attract high-achieving 
students who would not normally consider community college. 
Treat and Barnard (2012) also claim that honors programs facilitate 
regional efforts to “attract diversity in terms of underrepresented 
groups to their colleges and fulfill the promise of the traditional 
community college mission by making the transition from the 
community college to a selective four-year institution less onerous” 
(p. 695). Treat and Barnard (2012) add that community colleges 
serve more than half of all postsecondary students in the United 
States, many of whom are low income, minority, and/or first-gen- 
eration college students who face barriers to entry into selective 
colleges and universities. Mellow and Koh (2015) concur, stating 
that, “counter-intuitive though it may be—open-access commu- 
nity colleges need programs like honors to fulfill their mission of 
serving students who have been under-served and are under-repre- 
sented in higher education” (p. 66). The honors standard of offering 
small, learner-focused courses provides students the opportunity 
to establish a network of peer and faculty support, which substan- 
tially improves the prospects for successful completion (Mellow 
and Koh 2015). 


HONORS AND THE COMMUNITY COLLEGE MISSION 


Community college honors programs are not without skep- 
tics. Controversy abounds regarding whether two-year honors 
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programs contradict the egalitarian mission of the community 
college. In fact, some scholars have charged that honors pro- 
grams promote an elitist agenda. In response to Moltz (2010), who 
described a boom in community college honors programming, 
Shor (2010) of the CUNY College of Staten Island asks challenging 
questions: “Why not make the whole community college curric- 
ulum an Honors program? ... Democracy means a level playing 
field and equal protection for all, not tracking and privileging” 
After examining the pros and cons of community college honors 
programs, Floyd and Holloway (2006) concede the possibility that 
such programs potentially segregate high-achieving students from 
the regular student population, thereby creating an atmosphere of 
elitism; however, Floyd and Holloway (2006) ultimately conclude 
that offering honors classes actually allows community colleges to 
focus on social equality and level the playing field. Pruitt (2013) 
explains that honors contributes to social equality because insti- 
tutions offering honors meet the educational needs of students at 
every academic level, from the underprepared to the highly able 
and motivated student. 

Moreover, the presence of honors programs and honors stu- 
dents on campus can have other beneficial effects. Clauss (2011) 
points out that although honors students typically complete the 
majority of their general education requirements in honors, they 
take the majority of their courses outside of honors: “honors stu- 
dents typically take at least 75 percent of their coursework outside 
of honors. The influence of honors education beyond the perim- 
eters of a particular program is thus substantial as these bright 
students interact with their peers and teachers outside of honors” 
(p. 96). Heckler and Kanelos (2015) agree, stating that honors edu- 
cation enhances the experience of students not participating in 
honors because the traditional students benefit by observing and 
frequently embracing the honors students’ exceptional critical 
thinking and research skills. Honors students bring their appetite 
for engagement into non-honors classes across the curriculum, 
potentially revolutionizing classroom interactions by transforming 
class discussions into moments of uncertainty or surprise. Honors 
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students can conceivably inspire classmates to search for and find 
their own answers (Clauss 2011). From this perspective, the com- 
munity college honors program is actually serving all students, from 
those in learning support programs to those capable of the most 
exacting challenges. In fact, both faculty and staff recognize honors 
students as an important resource to leverage in efforts to facilitate 
community college student success. At Northeast State, honors stu- 
dents serve non-honors students as ambassadors, tutors, and peer 
mentors. Each semester, the College’s TRiO program, which is a 
Federal outreach program serving first-generation and low income 
students, and the College’s Center for Students with Disabilities 
recruit honors students to serve as tutors for their students. Since 
2015, Northeast State Honors Program students have also served as 
peer mentors to incoming Tennessee Promise students, who receive 
free tuition through Tennessee's last dollar scholarship program. In 
this role, honors student mentors ease Tennessee Promise students’ 
transition from high school to college and promote student engage- 
ment in and outside the classroom. Clearly, honors programs can 
significantly contribute to the achievement of the community col- 
lege mission and enhance the reputation of the institution. 


DATA ANALYSIS 


The purpose of this observational study is to compare the aca- 
demic achievement of community college students participating in 
honors programming (honors) to students who were academically 
eligible but did not participate in honors programming (non-hon- 
ors). Specifically, we test the following hypotheses: (1) there is a 
significant difference in final course grades for a required first-year 
writing course between non-honors and honors students; (2) there 
is a significant difference in grade point average two semesters after 
honors eligibility attainment between non-honors and honors stu- 
dents; (3) there is a significant difference in grade point average 
upon graduation between non-honors and honors students; (4) 
there is a significant difference in retention from fall of eligibility to 
fall of the second year (second-year retention) between non-honors 
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and honors students; (5) there is a significant difference in com- 
munity college graduation rate between non-honors and honors 
students; and (6) there is a significant difference in number of 
semesters to graduation between non-honors and honors students. 

We asked for and received Internal Review Board approval to 
access and analyze archival data collected from Northeast State 
Community College, a medium-sized community college in Ten- 
nessee. In sum, we included five honors-eligible cohorts from 
academic years 2008 to 2013 in the design. For the five cohorts, we 
collected the following data: (1) first-year cumulative GPA at the 
end of the second semester after honors eligibility (a minimum of 
24 credit hours); (2) cumulative GPA at the time of graduation; (3) 
retention from the fall of eligibility to the following fall (second- 
year retention); (4) graduation data, including number of semesters 
to completion; and (5) final course grade in English Composition 
II, a required first-year writing intensive course. The Office of Aca- 
demic Technology provided final course grades from Desire to 
Learn (D2L), the learning management system used by the institu- 
tion. For those students whose instructors did not utilize D2L, the 
Humanities Division staff provided grades collected from course 
records they routinely maintain. 

To be eligible for honors, new students must have earned a 
composite ACT score of 25 or higher or an SAT of 1140 or higher. 
Returning students became eligible if they achieved a 3.25 GPA or 
higher regardless of ACT or SAT scores. Of those who were eli- 
gible for honors, 95 participated in the honors program while 357 
did not. We present summary statistics describing the honors and 
non-honors comparison groups in Table 1. With a few exceptions, 
honors and non-honors groups were not substantively different. 
Among honors participants, 58 percent (n = 55) were female and 
42 percent (40) were males; of the non-honors students, 62 percent 
(223) were female, and 38 percent (134) were male. These results 
are somewhat different from those reported by Keller and Lacy 
(2013), who found that “women were more than twice as likely 
as men to participate in honors” (p. 78). Although the majority of 
honors-eligible students in the present study were female, a higher 
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percentage of females did not participate compared to the popula- 
tion of eligible males who did not participate. Eighty-seven percent 
of the honors students were white, whereas 95 percent of the non- 
honors students were white; there were not substantial numbers 
of other races represented within the data set to disaggregate spe- 
cific racial categories. Thus, we combine nonwhite groupings for a 
dichotomous measure of white compared to nonwhite race-ethnic- 
ity. Eighty-one percent of the honors students in the data set were 
traditional students, which we define as 24 years of age or younger. 
Nineteen percent of honors students were non-traditional students, 


Tasle 1. DemocrapuHics oF PoPuLATION 


Honors Non-Honors 

Measure N % N % 
Gender 

Female 

Male 
Race-Ethnicity 

Non-White 

White 
Age at Eligibility 

Traditional 

Non-Traditional 
Dual Enrolled 
High School Type 

Public 

Private 

Homeschool 
Low Income* 


First Generation * 


* The measures of income had missing data for 19 honors students (20%) and 41 non-honors students 
(11%). The measure of first-generation status had missing data for 47 honors students (50%) and 150 
non-honors students (42%). 
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defined as students 25 years of age or older. On the other hand, 76 
percent of non-honors students were traditional, while 24 percent 
were non-traditional. The vast majority (98%) of honors students 
had attended public high schools; similarly, 99 percent of non- 
honors students had attended public schools. Although none of the 
students attended private high schools, 2 percent of honors students 
were homeschooled, and 1 percent of non-honors students were 
homeschooled. Fifty-nine percent of the honors students met the 
criterion of low income status based on whether they received the 
maximum Pell award; likewise, 56 percent of non-honors students 
met the criterion of low income status. Interestingly, in the pres- 
ent study, a higher percentage of honors students than non-honors 
students were low income. Fifty-four percent of honors students 
were first-generation college students while 69 percent of non-hon- 
ors students were first-generation attendees. That 44 percent of the 
students in the data set left parental education information miss- 
ing and did not indicate parental education levels should be noted. 
Nonetheless, the majority of the students who did answer were low 
income and first-generation college students, characteristics associ- 
ated with students at risk of dropping out. 

Only 14 percent of non-honors students had participated 
in dual enrollment while 31 percent of honors students had par- 
ticipated in dual enrollment. Although both populations were 
likely to succeed in community college honors courses, the dual 
enrollment experience may have facilitated the development of 
self-efficacy in those students who decided to accept the honors 
challenge. Additionally, dual enrollment students may have devel- 
oped expectations of self that compelled them to take the honors 
challenge. Lile, Ottusch, Jones, and Richards (2017) found that dual 
enrollment students’ “sources of role expectations . . . included self- 
reflection and peer, family, teacher, and structural expectations” (p. 
95). A large percentage of honors participants were first-genera- 
tion college students (54%); however, a noticeably larger number 
of non-honors students (69%) were first generation. Similarly, the 
vast majority of both honors and non-honors students were white; 
however, while only 5 percent of non-honors students were non- 
white, 13 percent of honors students were nonwhite. 
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Honors Assessment Research Trends: 
Propensity Score Analysis 


Austin (2011) notes that, in observational studies, an individu- 
al’s decision to engage in a “treatment,” such as an honors program, 
is shaped by that individual’s attributes. Therefore, “baseline char- 
acteristics of treated subjects often differ systematically from those 
of untreated subjects” (p. 400). To address confounding variables, 
Austin recommends adjusting for fundamental variations when 
assessing the effect of a treatment on outcomes. Traditionally, 
researchers have utilized regression adjustment to explain differ- 
ences in baseline attributes between treated and untreated subjects, 
but researchers are increasingly interested in techniques grounded 
in the propensity score to diminish or remove, as Austin (2011) 
explains, “the effects of confounding when using observational 
data” (p. 400). Furtwengler (2015), however, warns of important 
limitations associated with establishing propensity scores regard- 
ing honors program participation and the associated outcomes; he 
recommends the following: (1) including students’ academic goals 
and declared majors as baseline characteristics, and (2) exploring 
the influence of honors and non-honors participation on individ- 
ual course success, graduation, retention, and time to graduation. 
Therefore, we utilized propensity score matching (PSM) to generate 
two equally matched sample groups that served as the foundation of 
the analyses. PSM utilizes logistic regression to generate a propen- 
sity score for individual participants, which indicates the probability 
that each individual will participate in the treatment under exami- 
nation: honors program participation in this case. In essence, the 
propensity score accounts for sample selection bias that contributes 
to differences in the probability of being in one group as opposed 
to the other (Grubb, Scott, and Good 2017). We utilized PSM as a 
method to better estimate the effect of honors programming, the 
treatment effect, on the student outcome criteria variables of grade 
in a first-year required English composition course; cumulative 
GPA two semesters after honors eligibility attainment; cumula- 
tive GPA upon graduation; second-year retention; graduation 
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rate; and number of semesters to completion between the groups 
(non-honors = 0, honors = 1). The propensity score was defined 
as the probability of honors participation based on the covariates 
listed in Table 1 because these characteristics impact academic suc- 
cess. Additionally, we included the baseline characteristics of ACT 
scores and high school GPA because this information provides “a 
measure of students’ motivation and perseverance” (Keller and 
Lacy 2013:76). 

We used archival data collected from the following years: 2008, 
2009, 2010, 2011, 2012, and 2013. For the five cohorts, the director 
of the college's office of Research and External Reporting collected 
all covariate and outcome data, resulting in an initial data set that 
included 4,931 individuals. We then screened the data set to remove 
students with incomplete covariate information such as no high 
school GPA, no or incomplete ACT score, incomplete Compass test 
score information, or no English Composition II grade. After we 
eliminated all of the incomplete or missing records from the data 
set, a total of 452 unique student records remained with 95 (21%) 
meeting the study’s definition of honors participants (honors) and 
357 (79%) meeting the study’s definition of honors-eligible non- 
participants (non-honors). 


Matching on the Propensity Score 


We then imported the data into R statistical software and 
matched students using the “Match It” package version 2.4-21 (Ho, 
Imai, King, and Stuart 2013). The Match It package contains several 
methods for matching and provides other packages to assist with 
analytical choices. Each individual received a propensity score in 
the data reports and a weight so that the covariates were balanced 
as evenly as possible. The propensity score signified the probability 
that an honors eligible student will enroll in honors based on the 
above 13 observable covariates, which represented the predictors. 
After matching, 95 non-honors students were matched with the 
95 honors students on the propensity score, leaving 190 students 
in the population for outcomes assessment. We then generated 
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a summary of the balance for the unmatched and matched data, 
which appears in the Appendix. 


Effect of Honors on Learning Outcomes 


In the next phase of the analysis, we determined the effects of 
honors participation on the above selected student outcome vari- 
ables. Furtwengler (2015) argued that researchers should ascertain 
the impact of the honors education experience so that “if a positive 
or negative effect [is] associated with participation . . . high-achiev- 
ing college-going students are aware of the associations and their 
options” (p. 275). We selected English Composition II as an out- 
come appropriate for assessment because the course is a general 
education core course that requires literary criticism, research, and 
analysis. Additionally, like Furtwengler (2015), we studied the scale 
of the impact of taking part in honors as measured by overall GPA 
because GPA has been correlated with “personality and motivation, 
achievement striving, individual learning, academic performance, 
[and] team learning...” (p. 279). We also sought to confirm Keller 
and Lacy’s (2013) conclusion that honors program participation is 
correlated with significantly higher second-year retention as well as 
a larger percentage of students who graduate in a timely manner. 
Because the study examines students who first enrolled in one of 
the fall semesters from 2008 to 2013, all the students included in 
the study had at least three years to complete a two-year credential. 
The graduation rate signifies Northeast State Community College 
graduation. 


RESULTS 


Concerning the hypothesis that a significant difference in final 
course grade for a required first-year writing course between non- 
honors and honors program participants exists, an independent 
samples t-test (t = 2.15, df = 186, p < .05) indicated that honors 
program participants (M = 91.18, SD = 6.41) were likely to earn 
significantly higher final course grades in English Composition II 
than their similar matched non-honors counterparts (M = 88.77, 
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SD = 8.81). Honors students were more likely to earn a final numer- 
ical course grade corresponding to the letter grade of A in English 
Composition II. Comparable non-honors students also performed 
well but were more likely to earn a final numerical course grade 
corresponding to the letter grade of B in the same course. The 95 
percent confidence interval for difference in means was .19 to 4.63; 
however, the Cohen’s effect size value (d = .31) suggests a weak to 
moderate practical significance of the difference. This finding con- 
firms Cosgrove's (2004) conclusion that honors program graduates, 
typically those students who are encouraged to ask intelligent and 
insightful questions, perform at the highest academic levels even 
when compared to students with equivalent ability. Table 2 illus- 
trates these findings. 

Table 3 presents the results of an independent-samples t-test to 
evaluate the hypothesis that a significant difference exists between 
honors and non-honors students in cumulative GPA two semesters 
after honors eligibility. The test was significant at the .001 level (t = 
4.42, df= 188, p< .001). Honors participants achieved a significantly 
higher cumulative GPA the second semester after honors eligibility 
(M = 3.71, SD = .35) than their comparable non-participant peers 
(M = 3.45, SD = .40). The 95 percent confidence interval for the 
difference in means was .13 to .35. Further, the Cohen’ effect size 
value (d = .69) suggests a moderate to strong practical significance. 
These results are consistent with Shushok’s (2006) findings regard- 
ing university-level honors students who earned a higher GPA 


TasLe 2. COompPositION II FinAL GRADES 


Program CI 


Honors* 6.41 [.19, 4.63] 
Non-honors 8.81 


*Two honors students withdrew from Composition II. 


Tasle 3. Cumutative GPA Two SEMESTERS AFTER ELIGIBILITY 


Program 
Honors 


Non-honors 
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than their non-honors counterparts by the end of the first year and 
suggest that community college honors participation has a posi- 
tive effect on academic achievement. These outcomes indicate a 
positive effect of honors education that high-achieving community 
college students should be encouraged to consider when weighing 
the options available to maximize their educational experience. 
Providing this information to students, who typically assume that 
honors participation will have a negative impact on their GPA, is 
especially important. 

Similarly, regarding the hypothesis that a significant differ- 
ence exists between honors students and non-honors students in 
cumulative GPA upon graduation, an independent samples t-test 
did reveal, in fact, a significant difference (t = 3.76, df = 142, p < 
.001). Results from this analysis are presented in Table 4. The aver- 
age cumulative GPA among honors students (M = 3.66, SD = .32) 
exceeds the minimum GPA necessary to achieve the institutional 
honor of cum laude (GPA 3.5-3.7), whereas the average cumula- 
tive GPA among non-honors students does not meet the minimum 
required for institutional honors (M = 3.44, SD = .38), which sug- 
gests that the honors experience has a positive impact on individual 
success and academic accomplishment. The 95 percent confidence 
interval for difference in means was .11 to .34, and the Cohen's 
effect size value (d = .63) suggests a moderate to strong practical 
significance. 

Concerning the hypothesis that a significant difference exists 
in second-year retention between non-honors and honors program 
participants, we utilized the Pearson chi-square test. Honors stu- 
dents were 11 percent more likely to persist one year after honors 
eligibility; the proportion of honors students who persisted to the 


Tasle 4. Cumutative GPA upon GRADUATION 


Program 


Honors 
Non-honors 


Note: Thirteen honors students did not graduate from Northeast State. Thirty-three non-honors 
students did not graduate from Northeast State. 
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fall of their second year after attaining eligibility was .89. The pro- 
portion of non-honors students who persisted to the fall of their 
second year was .80; however, honors program participation and 
second-year retention were found not to be significantly related 
(Pearson X” = 3.30, df = 1, N= 190, p = .07, Cramer’s V = .25). 

Regarding the hypothesis that a significant difference exists 
in community college graduation rates between non-honors stu- 
dents and honors students, honors participation and graduation 
were found to be significantly related (Pearson X* = 11.47, df = 1, 
N = 190, p < .001, Cramer’s V = .13). As illustrated in Figure 1, 
the proportion of honors students who graduated was .86 while 
the proportion of non-honors students who graduated was .65, 
suggesting that honors participation contributes significantly to 
community college degree completion. These outcomes indicate 
that honors students are 32 percent more likely to graduate than 
their non-honors peers. 


Figure 1. THREE-YEAR GRADUATION RATE FOR Honors AND 
Non-Honors ParticiPANTS 


Non-Honors Honors 
Participants 
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Table 5 indicates the results of an independent-samples t-test 
conducted to evaluate the hypothesis that a significant difference 
exists in number of semesters to completion between honors stu- 
dents and non-honors students. Although on first glance honors 
students (M = 6.35, SD = 1.82) appear to complete somewhat faster 
than non-honors students (M = 6.98, SD = 2.4), there was no sig- 
nificant difference in the number of semesters to completion (t = 
-1.73, df= 142, p = .08). The 95 percent confidence interval for dif- 
ference in means was -1.35 to .09. The Cohen’s effect size value (d 
= .30) suggests a weak to moderate practical significance. In both 
cases, honors and non-honors students complete community col- 
lege within the expected time frame at about the same pace. 


DISCUSSION 


The major findings of this study are that honors program 
participants (1) earned significantly higher final course grades in 
Composition II; (2) earned significantly higher cumulative GPAs 
the second semester after starting in the honors program; (3) earned 
significantly higher cumulative GPAs upon degree completion; 
and (4) were significantly more likely to graduate. These findings 
substantiate prior research and support increased investment in 
community college honors education as a high-impact educational 
practice particularly relevant to at-risk high-achieving students. 

Students often hesitate to take the honors challenge, perhaps 
because they do not possess accurate information about the ben- 
efits of honors. In an effort to increase community college honors 
participation, particularly among low income and first-generation 
students, honors directors might develop enhanced marketing 
strategies. Because these individuals are at-risk, they should be 
especially encouraged to pursue honors education. In particular, 


Taste 5. NuMBER OF SEMESTERS TO GRADUATION 


Program CI 


Honors [-1.35, .09] 
Non-honors 
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high-achieving at-risk students should be carefully informed of the 
benefits: higher course grades, higher GPAs, and higher graduation 
rates, even when controlling for baseline differences between honors 
and eligible non-honors students. When honors program directors 
request a list of eligible students, that list could include more com- 
prehensive data on eligible students, such as socioeconomic status, 
first-generation status, and veteran and disability status. With 
this additional information, honors directors can develop a more 
nuanced outreach. In general, invitations to join honors should 
include quantitative data illustrating the potentially positive impact 
on individual learning, motivation, and determination. Honors 
education often incorporates a number of what have come to be 
known as high-impact practices, such as writing intensive courses, 
undergraduate research, and vibrant learning communities. These 
results may well challenge existing myths and illustrate that honors 
participation has a positive impact on important measures of edu- 
cation outcomes. 


RECOMMENDATIONS FOR FURTHER RESEARCH 


This study examined the relationship between honors partici- 
pation and outcomes at a single community college using rigorous 
statistical methods to control for selection bias that influences who 
ends up in honors programs. To address this study’s limitation to 
one community college in Tennessee, increasing the scale would 
be worthwhile, for instance, by conducting a similar study of all 
community colleges offering honors programs in a given state. A 
comparative analysis of community college honors programs state- 
wide, including the structure of honors degree programs, staffing, 
extracurricular requirements, and measurable outcomes, would 
contribute to the further development and refinement of honors 
best practices. 

This study evaluated quantitative outcomes. In addition to 
evaluating quantitative outcomes, a survey of honors-eligible non- 
participants and honors participants regarding faculty interaction, 
extracurricular activity participation, and leadership development 
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activities would enrich future studies. Therefore, we recommend 
that future studies analyze information about the connection 
between community college engagement and academic success, 
particularly with regard to transfer scholarships. 

Honors programs would benefit from future research studies 
designed to discover why the majority of students eligible for com- 
munity college honors choose not to participate, particularly given 
the potential benefit to at-risk students. Specifically, a comparative 
analysis of honors participants and honors-eligible non-partici- 
pants across income and parental education levels would improve 
our understanding of why some students choose to take the honors 
challenge and why others decline. This analysis would also provide 
us with information we need to improve outreach to at-risk hon- 
ors-eligible students. 

All students at Northeast State Community College are required 
to take an exit exam before graduation. Access to the exit exam 
results, particularly critical thinking scores, for all honors students 
and honors-eligible non-participants would provide a crucial out- 
come variable to include as a measure of critical thinking ability, 
which is a major objective of honors education. Honors programs 
would benefit from a close examination of those scores as part of 
improving the quality of annual honors program assessment and 
reporting. 

Community colleges offering honors programs would clearly 
benefit from studying the impact of their programming, making 
improvements where indicated, and reporting the results to stu- 
dents and administrators alike in order to increase both investment 
and participation in honors programs. 
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Contributions of Small Honors Programs: 
The Case of a Public Liberal Arts College 


GEORGE SMEATON AND MARGARET WALSH 
KEENE STATE COLLEGE 


he Keene State College Honors Program began as the vision 

of a former college president to attract more high-achieving 
students to this particular public liberal arts college. In the fall of 
2007, after the college had secured initial funding, a small cohort of 
twenty first-year students were selected for the honors program by 
admissions staff for their achievements and promise. The numbers 
were intentionally small, but the goals were ambitious for a rural 
college that serves a high percentage of first-generation college 
students (43%). The students selected for admission into honors 
would enroll in an honors-level writing course and live together in 
a “parliament” inside one of the residence halls designed to link liv- 
ing and learning experiences. As second-year students, they would 
complete a global engagement faculty-led course that would cul- 
minate in immersive travel outside the United States. They would 
also complete several electives and a senior seminar that met their 
integrative studies requirements outside their major field of study. 
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Students would receive honors advising, tickets to selected arts and 
theater events on campus, and priority course registration. 

At the time of its inception, the idea of an honors program 
received mixed reviews from the faculty. Some were enthusiastic 
about the prospect of teaching these honors courses, and others 
opposed it in principle. Among the reasons for ambivalence were 
that channeling resources to students who came to the college hav- 
ing already demonstrated excellence could take away from average 
students who were yet to realize their potential. A decade later, the 
honors program continues to recruit, mentor, and graduate a small 
cohort of students. Students are now eligible to apply to enter the 
honors program in their second year, on the recommendation of 
faculty, a change that offers students a chance to find their stride in 
college before joining the program. Making this opportunity avail- 
able to more students resulted in expanding the cohort size. While 
small merit scholarships were initially guaranteed to honors stu- 
dents, the program has shifted to increasing its financial support 
for needy students, particularly in the global engagement course, 
which increases the tuition burden on students who are traveling 
by several thousand dollars per student. 

In 2017, after an external review and a change in leadership, 
the program was at a critical juncture as it began planning for the 
next decade. This study examines two issues that are important for 
assessing honors programs: (1) first-year to second-year student 
retention rates for high-achieving college students and (2) student 
engagement. 

The importance of attracting and retaining high-achieving 
students at institutions of higher education cannot be overstated. 
Demographic shifts have made the recruitment of college-ready 
students particularly challenging in the New England region. As 
Williams (2017) reported, New England's total number of new 
high school graduates is projected to decline by 14 percent by 2032. 
Colleges and universities are competing for students, and admis- 
sions offices are filling students’ mailboxes and email accounts with 
enticing amenities and tuition discounts. The governor of New 
Hampshire, for example, recently formed a committee of millennial 
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young adults to advise state leaders on issues facing them as they 
complete their education and prepare to enter the workplace (Asso- 
ciated Press 2017). While professional and career advising may 
ultimately be able to increase the number of workers and attract 
businesses, colleges may need to focus first on the retention of stu- 
dents through graduation. 

Numerous articles featured in the Journal of the National Col- 
legiate Honors Council have focused on honors student recruitment 
and retention. Kampfe, Chasek, and Falconer (2016) surveyed 
honors students at a state university and found that the two most 
important reasons students report for staying in their honors pro- 
gram are priority registration and the perceived prestige associated 
with honors membership. For students who were in their first two 
years, faculty-student connections, small high-quality classes, and 
a sense of a community were also significant factors. Goodstein 
and Szarek (2013) conducted a longitudinal study of retention 
and completion rates of honors students at a large public univer- 
sity. While students were very likely to continue from their first to 
second year—retention ranged from 88 to 90 percent—more than 
half of the students left later in the program, and completion rates 
ranged from 20 to 50 percent. The authors discussed efforts to 
improve program quality for students in midcareer, a time when 
undergraduates’ commitment to honors may waver as they focus 
on their major studies. For example, students in their second year 
were encouraged to engage fully in honors opportunities by “opting 
in, and these efforts appeared to increase completion rates. Michael 
K. Cundall (2013) argued that honors faculty need to show stu- 
dents that honors-level work is not synonymous with more of the 
same work but rather a new challenge. Close relationships forged 
in smaller classes with peers and professors allow honors students 
to do their best work. 

Many honors programs are informed by best practices from 
organizations such as the American Association of Colleges and 
Universities (AAC&U). In a 2008 report, George D. Kuh described 
the concept of high-impact educational practices for undergraduates 
(HIPs). Many of the effective teaching and learning strategies that 
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Kuh (2008) describes are course-based, such as first-year seminars, 
senior capstones, writing-intensive courses, and other intellectual 
experiences that consider core or big questions. Diversity, intercul- 
tural opportunities, and global learning offer students the chance to 
consider multiple viewpoints through study away, abroad, or in the 
local community. Supervised internships and faculty collaborating 
with students on research and service learning activities are also 
examples of HIPs. Kuh (2008) recommends encouraging all stu- 
dents to participate in at least two HIPs during their undergraduate 
career: one during their first year and one during their senior year. 
Although many majors and colleges have offered various forms of 
enrichment to students for years, the expansion of these opportuni- 
ties has coincided with a better understanding of the value for the 
kind of deep learning that comes from reflection and benefits all 
students. Clearly, these efforts can contribute to improved retention 
and skill acquisition. 

While high-impact practices may be designed for all under- 
graduate students, honors programs have been diligent about their 
efforts to integrate HIPs into the honors experience. Beginning in 
1994, the National Collegiate Honors Council has published “Basic 
Characteristics” for fully developed honors programs and honors 
colleges, and they have been updated periodically in response to 
changes in student needs and higher education (NCHC [1994] 
2010). Following the advice offered in the “Basic Characteristics,” 
honors courses in our program tend to have lower enrollment than 
other courses, and they are often spaces where innovations can be 
piloted for later use with a larger group of students. Ganesh and 
Smith (2017) used problem-based learning to enhance critical 
thinking and multidisciplinary learning in courses. They incor- 
porated collaborative and reflective approaches into the course 
design, and the instructors saw results in students’ improvement in 
grades and overall mastery of the course material in health courses. 
Banks and Gutiérrez (2017) found ways to “stack” study abroad 
with undergraduate research for social science students, which 
enhanced their preparation for graduate school and professional 
pursuits. 
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Given the importance of identifying means to improve reten- 
tion of an institution’s high-achieving students and the need to 
involve them in HIPs, the present study had three objectives. First, 
the study assessed the overall impact of honors program partici- 
pation on second-year retention. Second, it examined the effect 
of program participation on student engagement in HIPs. Third, 
through qualitative analysis of program documents, it examined 
honors program curriculum and instructional practices that may 
contribute to retention and student engagement. 


RESEARCH QUESTIONS AND HYPOTHESES 


Specifically, the study examined three broad research questions 
and tested two distinct research hypotheses. We enumerate those 
hypotheses below within the larger context of the research ques- 
tions that motivated the research. 


Research Question 1: 


What is the effect of honors program participation on sec- 
ond-year retention? 


Hypothesis 1: 


First-year honors program participants will have a higher 
retention rate than comparable non-honors first-year stu- 
dents who were awarded college-sponsored merit scholar- 
ships but who did not participate in the honors program. 


Research Question 2: 


How does honors program participation contribute to stu- 
dent participation in high-impact educational practices 
(HIPs)? 


Hypothesis 2: 


Honors program participants will be more likely than com- 
parison group students to report “Done or in progress” in 
response to the National Survey of Student Engagement 
(NSSE) (2011) items assessing involvement in HIPs. 
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Research Question 3: 


Which of the enriching educational opportunities offered by 
the honors program have an impact on students? 


Because this research question is exploratory and freewheel- 
ing, no hypothesis is proposed. 


ANALYSES OF STUDENT RETENTION 


We assessed the impact of honors program participation on 
retention by comparing second-year retention rates of program 
participants with rates obtained from a comparison group consist- 
ing of students who did not participate in the program but who 
had combined SAT scores that qualified them for honors program 
admission. 


Method 
Participants 


We obtained archival retention data from a sample of 2,383 
members of the incoming 2013-2015 fall cohorts of first-time, full- 
time, degree-seeking students. The sample consists of 984 men and 
1,399 women. The sample includes 53 honors program participants 
and 401 comparison group members. Like honors participants, 
comparison group members had combined math and verbal SAT 
scores of at least 1,100, the minimum SAT score required for admis- 
sion to the Keene State College Honors Program. 


Variables Examined 


We used archival institutional research data to test Hypotheses 
1 and 2. Membership in either the honors or comparison group 
served as the predictor variable. In addition, as a means of test- 
ing the similarity of the two groups, we examined three variables 
identified through previous internal institutional research at Keene 
State College as key predictors of retention: total scores on the Scho- 
lastic Aptitude Test (SAT), first-generation college student status 
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(having no parents who completed an undergraduate degree), and 
Federal Pell Grant eligibility. The latter variable serves as a measure 
of socioeconomic status. We defined and measured our criterion 
variable, retention, as returning to the same college during the fall 
semester one year after matriculation. We did not include students 
who may have returned two or more years after matriculation. 


Results 
Descriptive Statistics 


To assess the viability of the two predictor-variable groups, 
we compared the honors program group with the comparison 
group on a set of variables found to be predictive of retention at 
Keene State College. These included total SAT scores (i-e., math 
plus verbal), first-generation college status (having no parents who 
completed an undergraduate degree), and eligibility for a Federal 
Pell Grant. Table 1 presents a comparison of means for both hon- 
ors and comparable non-honors students. Honors participants had 
slightly lower SAT scores, and they were somewhat less likely to be 
first-generation or Pell-eligible students, but these differences were 
not statistically significant (Table 1). Thus, there is no evidence of 
pre-existing group differences in these variables that could account 
for differences in retention of students in each group. 


Test of Hypothesis 


Hypothesis 1 states: “First-year honors program participants will 
have a higher retention rate than comparable non-honors first-year 
students who were awarded college-sponsored merit scholarships 


Tasle 1. BENCHMARK CHARACTERISTICS OF Honors PARrTICIPANTS AND 
Comparison Group 


Variable Honors Participants | Comparison Group 
Mean SAT Total 1,164 (49) 1,169 (401) 


First Generation (%) 21.15 (52) 33.20 (401) 
Pell Eligible (%) 11.54 (52) 21.44 (401) 
Note: Numbers in parentheses are the number of cases. SAT total is the sum of math and verbal scores. 
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but who did not participate in the honors program.’ We tested this 
hypothesis using the chi-square statistical test. Chi-square (7) anal- 
ysis compares outcome frequency distributions (e.g., the frequency 
of those retained and not retained) across two or more groups to 
rule out the possibility that frequency differences observed across 
conditions are not the result of a chance occurrence. Higher y’ val- 
ues denote a lower probability (p) that frequency differences across 
groups can be attributed to chance. We used the p < .05 criterion 
for statistical significance commonly used in social science research. 
Figure 1 presents a graphic comparison of second-year retention for 
both groups. Among students in these two groups, those who began 
the year as members of the honors program were significantly more 
likely to be retained for a second year (y’ = 8.10, p = .004). Over 
94 percent of honors participants persisted to the second year, but 
only 81 percent did among the non-honors comparison group that 


Ficure 1. Honors ParticiPANTs AND ComMPARISON Group SECOND-YEAR 
RETENTION RATES 
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consisted of students who received college-sponsored merit scholar- 
ships based on their high school GPA but who did not participate in 
our honors program. 

As hypothesized, honors program participants were signifi- 
cantly more likely to be retained than were academically comparable 
non-honors students. Because this finding is based on pre-existing 
groups rather than random assignment to conditions, it is possible 
that pre-existing differences between the two groups in variables 
that were not examined in this study could account for this group 
difference. As noted above, however, the groups did not differ sig- 
nificantly in the three variables internal institutional research has 
identified as the best predictors of retention at Keene State College. 


ANALYSIS OF PARTICIPATION IN HIGH-IMPACT PRACTICES 


A key goal of the Keene State College Honors Program is to 
supplement classroom learning with enriching high-impact educa- 
tional practices (HIPs). Involvement in HIPs in one’s first year has 
been found to be predictive of first-to-second-year retention (Kuh 
2008). Therefore, the degree to which first-year honors students 
participate in HIPs may explain the program's positive impact 
on retaining highly prepared students for a second year. We used 
first-year NSSE data to determine if honors program participation 
results in differences in self-reported HIP involvement among first- 
year students. 


Method 
Participants 


NSSE data from 19 first-year honors program participants (16 
women and 3 men) were compared with responses from a compar- 
ison group of 102 first-year non-honors students (71 women and 
31 men) who received college-sponsored merit scholarships based 
on their high school GPA. Data from Keene State College students 
who completed the NSSE in 2014 and 2016 were combined for 
this analysis to provide an acceptable sample size for the honors 
and comparison groups. The gender breakdown for the honors 
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program group is not representative of the typical breakdown for 
honors program participants, which typically ranges from 55% to 
67% female. Although this discrepancy may represent a limita- 
tion to the discoveries obtained from the analyses of NSSE data, 
it is not likely to represent an alternative explanation of the find- 
ings because the female percentage of the honors and comparison 
groups did not significantly differ. 


Measures 


Administered to first-year and senior students, the National Sur- 
vey of Student Engagement contains 42 self-report items that assess 
four clusters of linked behaviors, experiences, and beliefs referred 
to by the instrument’s publishers as “engagement in activities that 
represent good educational practice” (Center for Postsecondary 
Research 2005:1). The engagement indicators include “academic 
challenge,” “learning with peers,’ “experiences with faculty,’ and 
“campus environment.’ Involvement in each type of engagement 
area has been found to have numerous positive academic outcomes 
(Astin 1993; Center for Postsecondary Research 2005; Chickering 
and Gamson 1987; Love and Love 2005; Pascarella and Terenzini 
2005). Findings from numerous studies attest to the measure’s reli- 
ability and validity (Kuh, Hayek, Zhao, and Carini 2002; Pascarella, 
Seifert, and Blaich 2010). 

To address our second research question, we examined NSSE 
items pertaining to participation in HIPs among first-year student 
respondents. Specifically, NSSE asks participants whether they had 
participated or plan to participate in each of a set of seven HIPs, 
including internships/co-ops/field experiences, participation in a 
learning community, study abroad, and collaboration with a fac- 
ulty member on a research project. In addition to providing data on 
each individual HIP, the survey output generates a global measure 
that reports the number of HIPs marked “Done or in progress.” 


Procedures 


Names and contact information for first-year and senior stu- 
dents were submitted to NSSE during the spring semesters of 2014 
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and 2016. The Center for Postsecondary Research administered the 
survey to a sample drawn from each class. The data file generated 
from the completed surveys contained unique student identifiers 
that we matched with campus data identifying honors and non- 
honors students to generate two independent groups: first-year 
honors program participants and a comparison group consisting 
of students included in the college's President's List and Dean's 
List who were not honors program participants. All students in 
the honors and comparison groups obtained merit-based schol- 
arships from the college. We used chi-square analyses of response 
frequency differences between the two groups to test hypothesis 2. 


Results 


Hypothesis 2 states: “Honors program participants will be more 
likely than comparison group students to report “Done or in prog- 
ress” in response to the National Survey of Student Engagement 
(NSSE) (2011) items assessing involvement in HIPs.” We found 
support for Hypothesis 2 in both the learning community HIP item 
and the global HIP measure. 


Participation in Learning Communities 


Figure 2 presents results for first-year students for the NSSE 
item asking about learning communities. Honors program partici- 
pants were more likely than comparison group members to report 
that they plan to participate in a learning community or have 
already participated in one (x” = 13.86, df = 3, p < .01). Among 
honors participants, 48 percent selected “Plan to do” or “Done or in 
progress” for this HIP; in contrast, only 32 percent of comparison 
group members did so (Figure 2). 


Global Measure of HIP Participation 


Figure 3 presents results for a global measure of high-impact 
practices done or in progress by the end of the first year of college. 
For the combined HIP measure, there was an even greater difference 
between the honors student and comparison groups (x” = 15.43, df 
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= 3, p < .001). Among honors participants, 73 percent achieved the 
goal of one first-year HIP, which was recommended by Kuh (2008), 
but only half as many in the comparison group (37%) did so. In 
addition to participating in learning communities, first-year stu- 
dents reported other HIPs as done or in progress; these included 
working with a faculty member on a research project (13%), intern- 
ships (7%), formal leadership in a student organization or group 
(6%), and study abroad (3%). 


FiGURE 2. PERCENTAGES FOR FIRST-YEAR Honors PARTICIPANTS AND 
Comparison Groups OF RESPONSES REPORTING EXPERIENCE 
WITH LEARNING COMMUNITIES 
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Source: National Survey of Student Engagement for Keene State College (2014 and 2016), 

Note: x? = 13.86, df = 3, p $01. 

Results are for those responding to a question prompt asking about “Learning Community or Some 
Other Formal Program Where Groups of Students Take Two or More Classes Together” There were 19 
honors participants, and there were 83 students in the comparison group. 
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Discussion 


Honors and comparison group differences in response fre- 
quencies for NSSE items provide some evidence that honors 
program participation may increase student involvement in HIPs. 
When compared with comparison group responses, honors partici- 
pants were more likely to participate in learning communities and 
other HIPs during their first year of college. In addition to pro- 
viding a means for explaining the program's impact on retention, 
Kuhs (2008) research on HIPs also suggests that such practices can 
increase graduation rates. It is possible, however, that students with 
an interest in participating in HIPs may also be interested in and 
qualified for the honors program. Therefore, additional research 


Ficure 3. NuMBER OF HicH-Impact Practices “Done or IN Procress” 
FOR Honors STUDENTS AND THE COMPARISON GRouP 
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is needed that surveys incoming honors and high-achieving non- 
honors students regarding their intention to take part in each of 
the HIPs examined in the NSSE. Responses to such measures could 
then be entered into a logistic regression equation to determine 
if honors participation accounts for a significant portion of HIP 
variance when controlled for pre-existing intent to participate. 
Additional research that follows honors students and similarly 
prepared non-honors students until they reach graduation is also 
needed. A longer time frame would also enable researchers to 
determine if this HIP effect persists until college graduation. 


IMPACT OF THE HONORS PROGRAM ON STUDENT ENGAGEMENT 


Finally, we used qualitative data to explore further the ways in 
which the honors program encourages student engagement. We 
tested the hypotheses above using quantitative data on student 
retention and involvement in HIPs. A limitation to this method- 
ology is that the findings do not provide evidence that outcome 
differences between participants and non-participants are directly 
attributable to the courses and policies of the colleges honors 
program. The retention and HIP differences could stem from dif- 
ferences in courses taken outside the program or in extracurricular 
involvement. In this final section, we used qualitative analysis of 
program documents to study how honors program curriculum and 
instructional practices may specifically contribute to retention and 
student engagement. The Keene State Honors Program is character- 
ized by several benefits as well as required components, including 
a first-year honors course, extracurricular events, priority registra- 
tion, and a common residential living community. This portion of 
the study looks at how faculty and students contribute to an experi- 
ence that may strengthen relationships among participants in their 
first two years. 


Method 


We conducted a qualitative review of honors program docu- 
ments from 2013, 2014, and 2015. These documents included 
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honors council meeting minutes, honors course syllabi, open-ended 
responses to questions on student satisfaction surveys, documen- 
tation of honors student activities, and events publicized on our 
campus website. We received approval from the campus Institu- 
tional Review Board to read and review these documents. From 
this review, we identified several components of the program that 
may shed light on higher retention and stronger student involve- 
ment for honors students early in their college experience. 


Results 


Research Question 3 asks: “Which of the enriching educational 
opportunities offered by the honors program have an impact on 
students?” Qualitative data allowed us to explore the answer to this 
question. 


First-Year Honors Course and Honors Housing 


In June 2013, the incoming program director invited faculty 
to attend a workshop to discuss their experiences teaching in the 
honors programs and their plans for the coming year. As Schuman 
(2006) advised in his handbook for honors program directors, 
honors programs need the very best faculty who will work with stu- 
dents effectively (see 27-28). When asked why they teach in honors, 
the professors said that they developed partnerships with the stu- 
dents, which created a more democratic classroom. They spent less 
time dictating rules and more time collaborating. The faculty also 
reported that they wanted to recapture the feeling of working with 
academically motivated students, to travel with students, to develop 
and enhance their research trajectory, to teach innovative material, 
to try new teaching and learning strategies, and to connect students 
to opportunities that they thought would benefit them. 

Faculty spent considerable time discussing what should be 
common experiences in the honors course electives. The topics that 
the group discussed were using active learning strategies, assigning 
comprehensive readings, allowing students to show class leadership 
with presentations throughout the semester, engaging in critical 
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dialogues, and making clear behavioral expectations for both stu- 
dents and faculty. 

In the fall of their first academic year, honors program students 
lived together and experienced the honors curriculum as a cohesive 
group. All Keene State students, including those in honors, must 
complete an introductory integrative studies course focused on 
thinking and writing in either the fall or spring semester. Keene 
State College designed this course as part of a general education 
requirement that included writing in stages, peer review, and indi- 
vidual conferences with the professor. The honors version of the 
course is offered only in the fall, and has had a profound impact 
in shaping students’ identity as honors-level learners because they 
take it at the beginning of their college experience. Moreover, the 
honors students have been living in honors housing together, mak- 
ing it easier for them to talk about their coursework outside of class 
and enhancing social connections among students. In essence, 
combining the course with a living-learning experience provided 
a ready-made mechanism for students who were seeking ways to 
connect on an intellectual level with other students outside of the 
classroom. That the resident assistant, typically an upper-level hon- 
ors student, plans extracurricular events to bring students together 
for a faculty panel, speaker, or theater performance enhances the 
experience of students living, working, and studying together. 

In 2013, for example, the theme for the introductory course 
focused on readings and discussions of how young people encoun- 
ter adulthood. The syllabus described assignments that students 
would complete as they worked toward writing a substantial 
research paper focused on a creative and multifaceted analysis of 
the transition to adulthood. Examples of topics that students wrote 
about included the meaning of maturity and responsibility, rites of 
passage in indigenous societies, emotional intelligence, and emo- 
tional literacy. 

In the spring of their first year, many students enrolled in 
more than one honors course in order to accelerate progress 
toward fulfilling their honors program requirements. Often they 
were encouraged and mentored by professors during small group 
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advising sessions. They also were likely to have individual conversa- 
tions with their honors resident assistant and the program director 
before enrolling in classes. Honors students benefitted from both 
general and focused advising in their first year. 

Syllabi of elective honors courses in the sciences, social sci- 
ences, arts, and humanities were specifically designed to encourage 
experimentation with new subject areas. For example, in a single 
semester Keene State offered first- and second-year honors students 
elective courses in astronomy, immigration, and intercultural com- 
munication. The syllabi contained language that conveyed common 
outcomes to students: their goal would be to demonstrate “an ability 
to transcend boundaries between experiential and classroom learn- 
ing” and “an ability to reflect upon and take responsibility for their 
continuing intellectual development.’ Professors constructed their 
own courses as they wished. There was no official template that hon- 
ors courses were required to follow; however, the written materials 
students received from honors professors conveyed a seriousness 
of purpose, a lengthy reading list, and a strong statement about 
classroom comportment and expected work ethic. During meet- 
ings of the Honors Council, a faculty body that provides oversight 
of the program, faculty members regularly addressed this impor- 
tant question: “What makes a course an honors course?” Each year 
the director issues a call to faculty members to join the Honors 
Council. The director also meets with prospective instructors who 
self-select to discuss how they might reframe an existing course or 
design a new experimental course for the honors program. 


Learning Communities 


The concept of learning communities is a HIP that closely fol- 
lows the philosophy and mission of colleges and universities with 
traditional-age students who live on campus. Bringing aspects of 
students’ social and academic lives together on a residential campus 
is one strategy to improve retention and success while enhancing 
the overall college experience. 

When the honors program was first designed at this institu- 
tion, attention was given to the overall experience of the students, 
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especially since faculty tended not to know the details of what goes 
on during the evenings and weekends beyond a general awareness 
of student parties and hall activities that occasionally include fac- 
ulty participation. In the early years of the honors program, the 
living and learning component was embedded into an existing 
residential program that created “parliaments” or specific areas in 
residence halls where students would choose to live together based 
on a common interest that could be academic, such as women’s and 
gender studies, or service-based, such as Habitat for Humanity. 
These communities were supported by programming and events 
that were largely planned by students, and their success was per- 
ceived as uneven and dependent on the energy and motivation of 
the particular students involved in a given group. The “honors par- 
liament” was distinctive because this choice of residence and room 
assignment was made for students after they applied and were 
accepted into the honors program. All first-year honors students 
were expected to be part of the honors parliament. Occasionally, 
students would request an exception to the residential component. 
Sometimes, an honors student who met a new friend during orien- 
tation or an athlete who would prefer to live with a teammate would 
ask to live somewhere other than the honors house. These requests 
were generally not granted. Exceptions were made only for com- 
muter students who opted to live at home during the first year. It is 
likely that some students did not join the honors program because 
of this requirement. Overall, however, the living experience of 
first-year students created the space for long-lasting friendships to 
flourish among students with common interests and goals. 

One improvement that brought stability and consistent pro- 
gramming to the honors parliaments was having an upper-class 
member of the honors program serve as a resident assistant. Resi- 
dent assistants also coordinate events such as outings to the theater 
followed by a panel discussion that includes honors faculty or eve- 
nings with the global education office staff, who share information 
and answer questions in the residence halls about study abroad 
opportunities prior to the application deadlines. The resident assis- 
tants help students deal with the travails and challenges of living in 
a residence hall and taking challenging honors courses. 
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In other words, resident assistants exercise their creativity and 
expertise in young adult development to bring together groups of 
students for a common purpose. 

Establishing strong connections among honors students, staff, 
and faculty in the first year of the program was beneficial. Prior to 
spring and fall registration, for example, the honors director and 
sometimes honors faculty members would visit the common space 
for individual or group advising meetings. These efforts paid off in 
helping to remind students of the courses available, encouraging 
students to speak with each other, and allowing the honors director 
to hear student concerns and recommendations for future courses. 
Given the small size of this honors program, this one-on-one com- 
munication was valuable, yet it required a great deal of effort on 
the part of the director to respond to individual scheduling needs. 
These encounters and activities helped to build an allegiance to the 
program capable of withstanding the heavy demands and workload 
that students faced as they progressed through their upper-level 
courses, embarked on internships, embraced study abroad, and 
pursued research opportunities. Academic enrichment opportuni- 
ties and residential life in higher education need not be mutually 
exclusive. On our campus and many others, these types of learn- 
ing experiences were wisely extended to non-honors and honors 
students alike, creating close, supportive relationships that improve 
retention and graduation rates for the entire student body. 


Discussion 


The Keene State College Honors Program purposefully inte- 
grates HIPs and best practices into its honors curriculum through 
both the design of its courses and its living-learning community 
component. Honors courses at Keene were designed specifically 
for this group of high-performing students. Experiential learning 
was integrated with traditional classroom instruction, and students 
were encouraged to accept personal responsibility for their educa- 
tion. In addition, through establishing learning communities where 
students reside together, students were able to apply outcomes from 
extracurricular activities to classroom curriculum, which results in 
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a richer overall academic experience. Although this arrangement 
did not eliminate the possibility that non-honors experiences could 
account for differences observed between participants and non- 
participants, it did provide evidence consistent with the idea that 
program elements directly contribute to increased retention and 
HIP participation. 

Although Research Question 3 focused on aspects of the hon- 
ors program and not on the experiences of the general student 
body or a comparison group of high-achieving students, drawing 
some inferences about the student experience on a college campus 
made up of several thousand undergraduate students is possible. 
While most of the general student body enrolled in introductory 
“Thinking and Writing” and “Quantitative Literacy” courses, the 
sequencing of first-year courses was not intentional, nor were advi- 
sors able to match courses to students’ needs until very recently. In 
contrast, the honors program has become a model for the entire 
campus. Beginning in 2016, a new student residence was opened 
that was designated for living and learning communities. All stu- 
dents were extended opportunities to select rooms in this dedicated 
space and to enroll in courses based on academic interests and 
themes. No evaluation of this model has been completed; however, 
as this model becomes established, opportunities to measure its 
impact on students’ persistence throughout their college careers 
and its effects on the larger campus environment will certainly be 
pursued. 


GENERAL DISCUSSION AND CONCLUSION 


This study examined outcomes associated with participation in 
an honors program at a small public liberal arts college. Among 
the most noteworthy of the outcomes examined was a significant 
increase in retention. Honors students were more likely to be 
retained for a second year than were comparable non-honors stu- 
dents. A second major finding was greater involvement in HIPs 
among honors participants than that reported by comparable non- 
honors students. Given Kuh’s (2008) finding that HIP participation 
contributes to retention, greater HIP participation among honors 
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students may have contributed to the higher retention observed 
among honors students. 

Nevertheless, because the quantitative component of this study 
did not utilize random assignment and a controlled experimental 
design, it is possible that differences between honors and non- 
honors students in coursework, extracurricular involvement, or 
some other factor could account for HIP and retention differences 
between the two groups. Although eliminating that possibility 
was beyond the scope of this study, the qualitative analysis of the 
program documents identified aspects of the curriculum and the 
learning community experience that promote HIPs, thus providing 
evidence that at least a component of the group differences in the 
outcome measures can be attributed to programmatic elements. 

Additional research on the outcomes of participation in a 
small honors program is needed to build upon the findings of the 
present study. Four specific approaches could yield important find- 
ings. First, because Kuh’s (2008) research on HIPs indicates that 
they contribute to both retention and graduation rates, parsing 
graduation data from honors and comparison students would be 
worthwhile. Second, another means of determining if program 
courses directly contribute to HIP participation would be to col- 
lect data from honors participants that assess their work in honors 
and non-honors courses. Third, the separate effects of living in a 
residential learning community and of the honors courses taken 
by first-year student participants could be examined by compar- 
ing the retention and HIP participation of three groups of students: 
honors students, comparison students living in a different learning 
community at the college, and comparison students not living in 
a learning community. The findings of the present study provide 
evidence of favorable outcomes from participation in the honors 
program and from specific honors program components. Although 
additional research is needed on the mechanisms underlying such 
outcomes, the current findings indicate that providing an honors 
program for high-performing students will yield benefits for the 
students who participate in it. Further, by increasing the retention 
of such students, honors programs will benefit the colleges and uni- 
versities that support them. 
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Demonstrating the Value of Honors: 
What Next? 


JERRY HERRON AND D. CARL FREEMAN 
WAYNE STATE UNIVERSITY 


ur professional organization, the National Collegiate Honors 

Council (NCHC), has provided a good general definition of 
honors education while at the same time recognizing the “diversity 
of honors experiences across many institutions of higher learning.” 
Here's how the definition reads, in part, from the NCHC website: 


Honors education is characterized by in-class and extra- 
curricular activities that are measurably broader, deeper, or 
more complex than comparable learning experiences typi- 
cally found at institutions of higher education. (NCHC 2013) 


Of crucial concern to the researchers in this collection is the quali- 
fier that honors education incorporates practices that are measurably 
superior. And as Smith (2019) points out, “With more than 1,500 
honors programs currently in operation and hundreds of millions 
of dollars being spent throughout American institutions, external 
pressure is building for accountability in honors programs” (p. 
27). (See also Scott and Smith 2016.) In response to the need for 
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accountability, our contributors have developed research to sub- 
stantiate in measurable ways the claims made on behalf of honors 
education and the application of what are frequently referred to as 
high-impact practices. No matter how convinced we may be per- 
sonally that honors adds value, it is essential to ourselves as honors 
educators, to our students, and to the constituencies we serve, both 
inside and outside our institutions, that we can support what we 
say with data, as Savage (2019) suggests in her contribution to this 
collection: 


Honors education is known nationally and internationally 
for leadership in high-quality undergraduate programs. 
Honors faculty enjoy the opportunity to create unique 
and innovative learning environments, with academically 
talented undergraduate students as the immediate benefi- 
ciaries. Institutions benefit from recruitment of ambitious, 
motivated students who typically have higher retention and 
graduation rates when compared to those in the traditional 
student population. Yet despite these obvious institutional 
benefits, questions persist regarding the value that honors 
adds and how precisely that value is to be measured. (pp. 
13-14) 


That is where the scholars and researchers in our volume contribute 
to the discourse—asking questions about the best practices for mea- 
suring “the value that honors adds” and the most effective means of 
representing these findings. Research in honors plays a vital role— 
that is how we justify our existence, it is how we learn from our 
mistakes and build on our successes, it is how we enlist students 
into becoming active participants in their own education—by dem- 
onstrating measurably and communicating effectively the value of 
what we do. 

To build on important work already done and to take account 
critically of the variables that will define honors research as we 
move forward, the contributors to this volume have undertaken a 
range of studies at institutions that differ in type from large research 
universities to liberal arts colleges to two-year colleges. And what 
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becomes clear is a consistent agreement about honors adding value, 
about the strategies and programs that work, and about the need 
for doing additional research to learn more. 

What is called for, then, and what our contributors have set out 
to provide, is a set of well-designed retrospective studies that assess 
students’ success quantitatively as they progress toward graduat- 
ing, comparing those who have participated in honors to those 
who have not. This research is not easily done because of the com- 
plexity involved in making sure we are comparing students who 
have the same level of preparation and motivation and who share 
other defining characteristics—comparing apples to apples as the 
cliché goes. Equally important is that we understand how students’ 
experiences are being changed qualitatively as well. Are the same 
practices and strategies equally effective and appropriate for every- 
one? Does one size fit all? Can practices be fine-tuned for different 
constituencies, whether defined by major or demographics or some 
other factors? The contributors to this monograph have set out to 
move the discussion of these important questions forward and also 
to speculate creatively as to what comes next. And what is also of 
critical importance is that they have undertaken to evaluate the best 
methods for creating and analyzing data, as well as the best means 
to communicate the significance of their findings. 

When it comes to quantitative measures, we might start with 
GPA and ACT/SAT scores, but honors educators generally agree 
that these figures are not providing all the data necessary when 
making decisions about who is admitted into honors and who is 
not. (These parameters often become the basis for group com- 
parisons as well after students have matriculated.) At Michigan 
Technological University, for example, in response to the perceived 
inadequacy of such measures, the Pavlis Honors College disregards 
traditional metrics altogether by making admission open to any 
student. What the Michigan Tech investigators found is that GPA 
was not telling them what they needed to know about measuring 
student performance. Their experience underscores the reasons for 
questioning traditional means of selecting students to join honors. 
What is called for, then, is perhaps a more creative way of thinking 
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about admissions criteria. An issue of the Journal of the National 
Collegiate Honors Council (14.2, 2013) was devoted partially to that 
very topic. But if we are not all going to follow the lead of the Pavlis 
Honors College, is there a better way of using GPA and ACT scores? 

Given that these are two data points we know about a great 
many of our students and that there is high probability that those 
scores will continue to be used, is there anything of value to be 
learned from them? Both are retrospective at the point of a student’s 
admission, so the question arises as to what predictive value they 
might have when it comes to future performance, and how the one, 
GPA, is related to the other, ACT/SAT. Some students’ high school 
GPA results reflect performance above what might be expected 
based on ACT/SAT scores; other students perform below expecta- 
tion. So, what can the relation of these two data points tell us about 
students once they arrive? Would it be possible to combine the two 
scores to produce a composite figure that might have greater pre- 
dictive value of student performance over time than either score 
on its own? And further, might our analysis be applied proactively 
to predict points at which a student with a given profile will likely 
encounter academic difficulty, and what kinds of intervention 
could we make before problems occur? And since most programs 
and colleges use additional measures, such as essays and interviews, 
in making admissions decisions and when awarding merit-based 
scholarships, is it possible to integrate all these different metrics, 
and if so, how? And what role might other factors play, such as lead- 
ership experience, extracurricular activities, or athletics, in helping 
us understand the likelihood of a student’s succeeding in honors? 
Clearly, there is more that we need to know. 

As to what—if any—use is to be made of standardized scores, 
that will depend on statistical analysis of honors students and their 
performance, which gets at the important matter of expertise. In 
order to conduct the kind of evaluations proposed in this volume, 
somebody would need to be versed in multivariate statistical tech- 
niques, and Bottoms and McCloud (2019) point out a potential 
difficulty in their study: 
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honors administrators, especially deans and associate 
deans, . . . often come from disciplines unfamiliar with 
multivariate statistical techniques. ... People who are new 
to statistics or use them infrequently might not understand 
how to answer various questions using the proper analysis 
or the proper statistical controls. (pp. 52-53) 


Given the ubiquity and—perhaps mistaken—primacy traditionally 
accorded to college entrance tests and GPA as measures, as well as 
the bragging rights attached to both by administrators when it comes 
to demonstrating the rigor or the quality of an honors program or 
college to prospective students, it is probably worth devoting some 
careful attention and statistical rigor to thinking through the ways 
these measures are to be used and of course why and how. And it 
is also worth giving some serious thought to explaining why such 
measures are lacking individually and what is to take their place. 
Or, for that matter, whether GPA is a useful measure at all, on its 
own, of students’ success once they enter an honors program or col- 
lege, which is a question that Meadows, Hollister, Raber, and Fiss 
(2019) raise in their study, proposing that “college GPA remains a 
limited measure of a certain type of success and that this measure 
is not necessarily predictive of success in postgraduate endeavors” 
(p. 117). 

The question of which measures to use and why returns things 
to the matter of multivariate statistical analysis and the need for 
it, which is a point that Diaz, Farruggia, Wellman, and Bottoms 
(2019) make: 


Considerable research to date on the impact of honors 
education lacks the appropriate controls to account for 
alternative explanations for the differences often observed 
in the success of honors versus non-honors students. (p. 60) 


The consequence is that evaluative findings suffer from serious lim- 
itations, as Cognard-Black (2019) suggests: 


Thus, the evidence most often used to demonstrate the 
impact of honors programs is limited because it usually 
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does not account for the differences that exist between 
honors and non-honors students at the moment of matric- 
ulation or point of entry into honors programs. That reality 
makes it difficult to establish a causal connection between 
the honors experience and student change. .. . (p. 5) 


What we want to know is the measurable difference made by hon- 
ors programming; we want to determine which specific practices 
contribute to differences in the performance of comparable honors 
versus non-honors students, eliminating as many alternate expla- 
nations as possible. Otherwise we will find ourselves without a 
compelling answer to the objections that honors students are sim- 
ply good students to begin with and that they would do well no 
matter what, honors or no honors, which makes justifying our exis- 
tence at budget time a great deal harder. The contributors to this 
collection offer clear demonstrations of what rigorous value added 
analyses will require and how they can be accomplished. 

The work of Spisak, Kirby, and Johnson (2019) is critical to 
this enterprise. They set out to address a gap in current research by 
evaluating the effect on academic performance of honors housing 
and a pre-semester elective class taken by entering honors students 
at the University of Iowa: 


As with first-year seminars, much scholarship exists on the 
effects of residence halls and living-learning communities 
on the success of students. . . . Little comprehensive data 
have been collected, however, specifically on the effects of 
the honors residence hall experience on students’ academic 
outcomes. ... (p. 153) 


Based on the Iowa investigators’ positive results, knowing if other, 
similar community-building activities might also play a role in stu- 
dents’ academic success and whether the same results would follow 
at other kinds of institutions would be important. 

In other words, there is much that we do not know—yet. And 
this same gap applies not only to quantitative analysis, but to 
qualitative measures as well. As Spisak, Kirby, and Johnson (2019) 
point out: 
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It could be that orientation-like experiences benefit stu- 
dents in ways that are not normally tracked, such as their 
effect on alleviating the anxiety associated with transition- 
ing into the university... . Such benefits may not always 
show themselves through GPAs, engagement in the pro- 
gram, and persistence, and yet they may well be valuable to 
students in other ways. ... (p. 174) 


Finally, it is not all a matter of multivariate, quantitative data. As 
Smith (2019) points out, a comprehensive assessment of student 
learning and honors value added will require “the use of both quan- 
titative measures, such as student grades or credit hours earned, 
and qualitative measures, such as the review of a portfolio or cap- 
stone project” (p. 31). We have much to learn about the other ways 
in which honors is adding value—ways not necessarily subject to 
quantifiable analytics. 

Meadows, Hollister, Raber, and Fiss (2019) raise this point as 
well in their application of the theory of “self-authorship,’ described 
by Baxter Magolda (2008) as “‘the internal capacity for an individ- 
ual to define one’s beliefs, identity and social relations” (quoted in 
Meadows et al. p. 119). Their investigation offers 


insight into the potential for a written reflection protocol to 
be used as an assessment for self-authorship. While more 
work is needed, the results shown here suggest that focusing 
our honors college on specific learning goals and using these 
as measures of success other than GPA provide a framework 
for our curriculum and assessment and also create an envi- 
ronment in which students may find a deeper connection 
between their self-defined future and their coursework such 
that GPA becomes a product of engagement with the hon- 
ors college rather than a measure of potential for success. 


(p. 143) 


Particularly suggestive here, relative to the kinds of investigations 
that might come next, is the connection between quantitative and 
qualitative outcomes, and how the one, such as GPA, might become 
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a product of the other, rather than being a stand-alone measure 
in itself. The question is what precisely the GPA is measuring and 
whether there might be alternative, more comprehensive means of 
evaluating students’ performance. 

Clearly, we do not know nearly as much about qualitative value 
added as we know about quantitative measures. For instance, it 
would be useful to have data showing how individual students are 
changed as they move through an honors curriculum, not in rela- 
tion to non-honors students, but in relation to their own starting 
points—changes such as those suggested by the investigation of 
self-authorship. The study done by Smeaton and Walsh makes a 
valuable contribution here, relative to qualitative value added and 
the work they have undertaken to understand high-impact educa- 
tional practices (HIPs) for undergraduates at a public liberal arts 
college: “through qualitative analysis of program documents, [the 
study] examined honors program curriculum and instructional 
practices that may contribute to retention and student engage- 
ment” (p. 233). Particularly valuable is their use of National Survey 
of Student Engagement (NSSE) data in conducting their study: 
“Honors and comparison group differences in response frequen- 
cies for NSSE items provide some evidence that honors program 
participation may increase student involvement in HIPs” (p. 241). 

In this connection, the Research Committee of NCHC has 
proposed a step forward in partnership with the National Survey 
of Student Engagement (NSSE)—a partnership that allows for the 
addition of questions to the NSSE surveys distributed on partici- 
pating campuses. A similar project is currently in development 
in conjunction with the Community College Survey of Student 
Engagement (CCSSE). The prospect of such results leads one to 
wonder additionally about the post-baccalaureate lives of our stu- 
dents and whether honors graduates become critical thinkers, find 
job satisfaction, or engage in lifelong learning. Are our graduates 
more likely to become active members of their communities? Such 
questions are important, as Diaz, Farruggia, Wellman, and Bottoms 
(2019) suggest (p. 86). But these are factors about which we know 
comparatively little, and, admittedly, it would be no easy matter to 
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develop data to answer those questions. But our mandate to make 
a measurable difference in students’ lives surely suggests that we 
ought to try. 

Regardless of how much good data we collect, another prob- 
lem needs to be solved: how to report results in an appropriate 
and persuasive form. Here, we might take a lesson from English 
Composition 101: the usual instruction to students is that they 
need to know their audience if they are going to write an effective 
essay, particularly one that is intended to persuade. When it comes 
to honors and value added, not all audiences are the same; some 
need more complex, data-driven explanations than others. But it 
is probably safe to assume that starting with something simple and 
understandable is the best way to proceed. To that end, there are 
the questions of why students choose one college or university as 
opposed to another, and whether honors had anything to do with 
that choice. Simply asking what role honors has played in deci- 
sion-making is easy; Bottoms and McCloud (2019) report that the 
University of Illinois at Chicago (UIC) asked this question of their 
first-year students: ““Would you have come to UIC had it not been 
for the Honors College?’” (p. 43). Their results prove persuasive and 
easy to communicate: “Fully 65 percent of all honors students and 
75 percent of our most prestigious diversity scholarship students 
said ‘no’” (p. 43). That students would not have chosen to attend a 
particular institution had it not been for honors is certainly strong 
and compelling evidence of value added. Brown, Winburn, and 
Sullivan-Gonzalez (2019) undertake a similar analysis, and with 
similarly positive results at the University of Mississippi relative to 
the value honors adds in recruitment. 

The question, then, is how to make best use of what we measur- 
ably know. The answer might be thought of in terms of value added 
factors, which could refer to a whole range of potential points of 
special pride. Imagine being able to tell prospective students and 
parents that an undergraduate honors student's time to degree, or 
cumulative GPA, or likelihood of gaining admission to graduate 
or professional school, or job placement is improved by a spe- 
cific multiplier or value added factor. That would be a clear way 
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of communicating a possibly complex data analysis. Or imagine 
being able to tell a college or university president or the head of the 
development office that a positive value added factor for honors 
graduates can predict those who are more likely to become donors 
to an institution by a certain percentage. Those bits of information 
would all be persuasive for any honors dean or director to marshal. 
But like many things that may seem easy, they require a good deal 
of thinking ahead and planning useful assessment strategies before 
the occasion arises when we are called on to demonstrate the value 
that honors adds. 

Another possibility, related to the survey of students’ likeli- 
hood of choosing a particular school, is what might be called the 
“halo effect.” As the two studies just referred to have shown, there 
is a halo effect relative to honors; Brown, Winburn, and Sullivan- 
Gonzalez (2019) write: “Our data reveal the honors college to be 
a significant component in the decisions of Mississippi's highest- 
achieving students to attend the [University of Mississippi]. One 
significant additional consequence is that attracting outstanding 
students from other states has a strong impact on the diversity of 
the university student body” (p. 190). Those results are suggestive 
of what more we have to learn and whether similar instances of 
the honors halo effect might exist on other campuses. The better 
we understand the appeal of honors—the halo effect—the better 
able we will be when it comes to targeting recruitment efforts to 
specific student populations, quoting again from Brown, Winburn, 
and Sullivan-Gonzalez (2019): 


Students who apply to the honors college may well possess 
traits that differentiate them from their academically simi- 
lar counterparts who do not apply, and these traits may be 
related to retention. At the same time, however, it is also 
likely that the honors environment that attracts these stu- 
dents in the first place is also successful in providing them 
with an academic experience that fosters the intellectual 
and personal growth that they seek and that the honors 
environment and experiences translate into increased aca- 
demic success and retention. (p. 198) 
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As this conclusion suggests, we need to know more about the stu- 
dents who choose honors and their motivations for making that 
decision as well as more about how they compare with students not 
in honors. 

The halo effect might well extend to those other, non-honors stu- 
dents as well. For example, does the existence of a high-profile honors 
program or college demonstrably contribute to an institution's over- 
all prestige and recruitment potential? Given the remarkable growth 
in honors education, particularly the growth in the number of hon- 
ors colleges (Scott and Smith 2016), is there a correlation between 
the inception of an honors college at a particular institution and 
positive changes in the demographics of applicants overall? Even if 
a student does not choose honors, does the existence of a high-pro- 
file honors program influence student decision-making generally? 
That would be interesting to know. And is there a halo effect when 
it comes to establishing a critical mass of engaged honors students, 
which is a question suggested by the work of Spisak, Kirby, and 
Johnson (2019) in their study of residence halls and pre-college 
experiences? At what point, and in what measurable ways, might the 
presence of a specific population—a critical mass—of engaged hon- 
ors students begin to produce added value above and outside of the 
programmatic elements that bring them together? How many stu- 
dents are needed to constitute a critical mass, and is it the same for 
all types of institutions? And do all students benefit equally, STEM 
students versus humanities majors, for instance? Do our practices 
benefit students equally regardless of their level of preparation and 
motivation? And is the honors offer equally attractive across dif- 
ferences of demographics? And if not, how do we make up for the 
deficits? 

Of particular interest here is the student invited into honors 
who declines the invitation; Honeycutt (2019) points out: 


Honors programs would benefit from future research stud- 
ies designed to discover why the majority of students eligible 
for community college honors choose not to participate, 
particularly given the potential benefit to at-risk students. 
Specifically, a comparative analysis of honors participants 
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and honors-eligible non-participants across income and 
parental education levels would improve our understanding 
of why some students choose to take the honors challenge 
and why others decline. (p. 220) 


Do similar patterns exist across the board at different kinds of insti- 
tutions? And are all students who decline the invitation the same? 
Or are there differences with respect to demography, academic 
major, and STEM versus non-STEM, and what about measures 
such as academic preparation and motivation? Is there any relation 
between a student's likelihood of declining and the potential benefit 
of the program? In other words, are students who stand to benefit 
the most possibly the most likely to decline honors? That informa- 
tion would be important to know. How do we understand their 
decision-making, and how do we use evidence relative to retention, 
academic performance, and graduation to persuade those students 
of the value of honors? 

To better understand these variables, Honeycutt’s study (2019) 
uses propensity score analysis as a useful analytical tool: “The 
propensity score signified the probability that an honors eligible 
student will enroll in honors based on . . . 13 observable covari- 
ates, which represented the predictors” (p. 213). The 13 covariates 
include such data points as high school GPA, dual enrollment status, 
ACT scores, income level, first-generation status, age, and gender 
(pp. 210, 213). Honeycutt offers the following practical conclusion 
relative to the use of positive benefit factors that might persuade a 
student to choose honors: 


Students often hesitate to take the honors challenge, per- 
haps because they do not possess accurate information 
about the benefits of honors. . . . In particular, high-achiev- 
ing at-risk students should be carefully informed of the 
benefits: higher course grades, higher GPAs, and higher 
graduation rates, even when controlling for baseline dif- 
ferences between honors and eligible non-honors students. 
When honors program directors request a list of eligible 
students, that list could include more comprehensive data 
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on eligible students, such as socioeconomic status, first- 
generation status, and veteran and disability status. With 
this additional information, honors directors can develop a 
more nuanced outreach. (pp. 218-19) 


We might reasonably ask what qualitative data we could bring to 
bear relative to a student’s experience and how honors makes that 
experience more satisfying and worth pursuing across a range of 
differences that characterize our students, including veterans and 
students with disabilities. 

Implicit here is a highly suggestive point about honors and 
diversity. As Brown, Winburn, and Sullivan-Gonzalez (2019) have 
shown, honors helped achieve geographic diversity at their insti- 
tution. And Honeycutt’s study suggests a strong, positive role that 
honors might play in strategically recruiting and graduating at-risk 
students. Diaz, Farruggia, Wellman, and Bottoms (2019) make a 
similar point with respect to underrepresented students: 


this study shows that honors education has a statistically 
significant positive effect on student success above and 
beyond all other background characteristics studied. . . . 
Furthermore, and of great importance in a nation where a 
significant gap in the success of underrepresented students 
versus others exists, we found that the positive effects of 
honors college membership were more pronounced for 
African American and Latino/a students for some indica- 
tors of success. (p. 79) 


Not only does honors work, with measurable positive benefits, it 
works particularly well for certain populations of students. 

Thus, when it comes to promoting diversity, honors is anything 
but an extravagance or an elitist enterprise. On the contrary, honors 
is a driver for achieving positive results, and the more students who 
take part, the greater the benefit. That insight leads to the quite rea- 
sonable conclusion proposed by Patton, Coleman, and Kay (2019): 


The data collected here [from Eastern Kentucky Univer- 
sity] show honors students outperforming the comparable 
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non-honors group in measures of second-year retention 
and four- and five-year graduation, regardless of pre-col- 
lege academic preparation. ... The impact on a university's 
retention and graduation rates would be profound if more 
students were exposed to the honors program environ- 
ment. In an era of public scrutiny and with the proliferation 
of performance-based funding . . . [,] making the case to 
high-level university administration that honors educa- 
tion positively impacts these metrics [such as retention and 
graduation rates] for its students is extremely beneficial for 
honors deans and directors. (pp. 110-111) 


In this context honors clearly becomes a laboratory for testing best 
practices, finding out what works and what does not work, and then 
sharing results to promote better outcomes for all our students. 

Relative to their program and the application of lessons learned 
beyond the honors population, Smeaton and Walsh (2019) point 
out that “the honors program has become a model for the entire 
campus” (p. 248). There is much to be said for making friends by 
sharing rather than hoarding successful high-impact practices and 
thus countering the frequent objection that honors is an elitist 
undertaking not relevant to the experience of most students or fac- 
ulty. Not every student is going to be in honors, or want to be, and 
the same holds true for faculty, but what can we do, what practices 
can we share, to make life better for everyone? Although the pieces 
in this collection concentrate on student success, it will be impor- 
tant for future work to ask questions about the value that honors 
adds for faculty development and retention and the role honors 
programs and colleges can play in promoting curriculum develop- 
ment and helping to achieve institution-wide learning outcomes. In 
other words, measuring the value that honors might add needs to 
happen in a variety of areas and contexts. 

When it comes to institution-wide benefits, some honors 
practices are labor-intensive and expensive, relatively speaking, 
but others are less so. For example, living and learning communi- 
ties can be created by mobilizing existing resources and following 
honors models to deliver a positive benefit to a larger population. 
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What we could use more of at this point is a kind of bottom-line 
thinking and self-representation. If our high-impact practices pro- 
duce positive results—particularly with respect to such measures as 
retention, credit hours passed, bounce-back from probation, and 
time to graduation—is it possible to translate those outcomes into 
dollars and cents? If we can workshop ideas to improve retention 
generally, for instance, or to decrease a student's time to degree, 
what do these mean with respect to tuition dollars paid back to the 
institution or savings to students and parents achieved by decreas- 
ing the time an undergraduate spends paying for a degree? And 
what about students who join honors in progress? Often, programs 
and colleges offer more than a single kind of honors regimen, with 
tracks that are not mutually exclusive: one, a comprehensive, gen- 
eralized track that begins in the first year and continues through to 
graduation; and another, discipline-specific track that leads to hon- 
ors distinction in a major. Are there value added benefits particular 
to students who are not enrolled in honors from first year through 
graduation, and how does their performance compare with other 
students—those not in honors, or those who complete a full honors 
curriculum? Are the benefits of honors participation cumulative? 
The findings of Diaz, Farruggia, Wellman, and Bottoms (2019) are 
suggestive at this point: 


although honors college participation at any point in the 
students’ college careers led to a higher chance of graduat- 
ing in four or six years, the more time students spent in this 
honors college, the more successful they were in terms of 
the likelihood of graduating. (p. 84) 


And what do we need to know about value added and students who 
matriculate by way of transfer agreements that link two-year to 
four-year institutions? How are high-impact practices best shared 
across those institutional boundaries? 

As this brief review and the papers assembled here make clear, 
we know a good deal already. At the same time, we still want to 
know even more. To that end, the present collection is an invitation 
to further research rather than a last word. For instance, as Bottoms 
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and McCloud (2019) point out, even though analyses provide evi- 
dence of the effectiveness of honors, a number of questions remain: 


even though the analyses support the contention that hon- 
ors education is effective, they do little to explain why. .. . 
Further, it is important to identify which practices are best 
for which students. This information could lead to under- 
standing why the effects of honors experiences are stronger 
for students of some races/ethnicities compared to others. 


(pp. 51-52) 


Recognizing why honors programs work the way they do, Diaz, 
Farruggia, Wellman, and Bottoms (2019) provide useful sugges- 
tions for further investigation: 


Future research could expand the definition of student 
success to include elements such as lifelong learning, later- 
life civic engagement, graduate and professional school 
matriculation and success, or career development, and it 
could begin to tease apart the various features of the hon- 
ors experience that contribute most to student success, 
with qualitative and quantitative methods. Future research 
should also continue to identify factors that explain student 
success of both honors and non-honors students. (p. 86) 


And it is not just honors students and faculty that we need to study 
and learn more about; there are honors administrators as well, 
which is a point that emerges from Smith’s study: 


only 31 percent [of survey participants] say that outcomes 
assessment data are actually being used to guide the majority 
of program changes. This finding demonstrates that honors 
deans and directors are struggling to apply the skills they 
have to “close the loop” and effectively apply assessment 
practices for the process of continuous improvement. (p. 37) 


In other words, we have plenty of good and interesting work ahead 
of us, which will call for creative collaborating and coordinating 
among colleagues, the Research Committee, and our NCHC office 
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as we take our next steps. By way of a conclusion, which is really 
more of an invitation, we offer the following ideas for what to do 
next, given what we now know and what we want to know. 


ie 


Create an online means for honors researchers to make oth- 
ers aware of ongoing research in order to share results and 
collect data across institutions. 


. Explore the possibility of creating a web location sponsored 


by NCHC for working papers that report results and share 
ideas quickly, with the end goal of formal, peer-reviewed 
publication. 


. Pursue collaborations with the Center for Postsecondary 


Research (which administers the NSSE), the Center for 
Community College Student Engagement (which admin- 
isters the CCSSE), and other higher education researchers; 
gather results; and expand qualitative analyses to support 
quantitative studies. 


. Explore the possibility of an experts-on-demand resource to 


provide deans and directors who are not experts in multi- 
variate analysis the help they need. 


. Create an online toolkit for honors researchers, particularly 


those new to their jobs, to provide show-and-tell advice 
about presenting what we know and how best to commu- 
nicate results; make a part of that toolkit best-practices 
applications that can readily be deployed. 


. Make sure that colleagues are aware of NCHC resources for 


finding and contributing to research: JNCHC, HIP, and the 
National Collegiate Honors Council Monograph Series; and 
share information about accessing NCHC’s searchable indi- 
ces as well as other searchable databases relevant to honors 
research. 


. Invite fellow researchers to help us learn more about 


a. GPA and ACT/SAT scores, and what if anything we have 
to learn from these measures; 
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b. Honors completion, and why/when students stop work- 
ing toward honors graduation requirements; 


c. Honors advising and how we measure success; 


d. Qualitative value that honors adds to students lives and 
experiences after they graduate; 


e. Two-year to four-year transfers and how to manage them; 


f. Honors populations we want to know more about, such as 
veterans, students with disabilities, etc.; 


g. The value that honors adds for faculty relative to retention 
and faculty development; 


h. Honors as a driver for curriculum development; and 


i. Honors administrators and best-practices. 
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Call for Paper Proposals 


Research on the Benefits of Honors Program Participation 


NATIONAL COLLEGIATE HONORS COUNCIL 
MONOGRAPH SERIES 


Deadline for Submissions: May 1, 2017 
Direct Submissions and Inquiries to: Dr. Jerry Herron, jerry.herron@wayne.edu 


At this time, we invite submission of proposals for an upcoming volume featuring new 
empirical research exploring the value added of honors program experience, to be published 
in the National Collegiate Honors Council Monograph Series in 2018. It is now routine for 
many in honors administration to offer student accomplishments vis-a-vis those of non- 
honors students as evidence that honors programs are successful, vital components of the 
colleges and universities where they are housed. Many programs can show that their honors 
students graduate at higher rates than non-honors students, that they graduate sooner, that 
they graduate with higher GPAs, that they are more likely to go on to graduate and profes- 
sional school, and that they win prestigious fellowships at higher rates. Most honors students, 
however, are starting at a different place than students in the general student body because 
admissions processes for most honors programs and colleges ensure that unusually smart, 
talented, and highly motivated students enter their programs. Thus, the evidence often used 
to demonstrate the success of honors programs is limited because it does not statistically 
control or otherwise account for the differences that exist between honors and non-honors 
students at the moment that they matriculate and enter into honors programs. We seek proj- 
ects using methodologically rigorous approaches to disentangle the effects of honors program 
participation from baseline student characteristics, but we also encourage researchers using 
qualitative or mixed methods approaches to illustrate in creative ways the unique effects of 
the honors program experience and how the criteria used for selecting honors students might 
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themselves be interrogated. We especially encourage proposals that also address any varying 
effects of the honors experience for those with different gender identities or for first-gen- 
eration students, racial or ethnic minorities, or other groups that face unique challenges in 
higher education. Proposals should include detail about the sources and kinds of data used; 
verification, when necessary, that appropriate IRB approval has been secured before collec- 
tion or use of data; what research method(s) will be employed; what stage of development 
the project is in currently; and either actual or expected findings from the research. Those 
wishing for their work to be considered for inclusion should submit proposals of 250-500 
words by May 1, 2017; manuscripts for accepted proposals will be collected in October 2017, 
with anticipated publication in 2018. Send proposals to Dr. Jerry Herron, Dean of Honors 
College, Wayne State University (email: jerry.herron@wayne.edu). Address inquiries to the 


same. www.nchchonors.org/monographcall 
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best practices, and philosophies that illustrate how to evaluate an honors thesis program, solve pressing problems, select effective requirements 
and procedures, or introduce a new honors thesis program. 
Housing Honors edited by Linda Frost, Lisa W. Kay, and Rachael Poe (2015, 352pp). This collection of essays addresses the issues of where 
honors lives and how honors space influences educators and students. This volume includes the results of a survey of over 400 institutions; 
essays on the acquisition, construction, renovation, development, and even the loss of honors space; a forum offering a range of perspectives 
on residential space for honors students; and a section featuring student perspectives. 

If Honors Students Were People: Holistic Honors Education by Samuel Schuman (2013, 256pp). What if honors students were people? 
What if they were not disembodied intellects but whole persons with physical bodies and questing spirits? Of course . . . they are. This 
monograph examines the spiritual yearnings of college students and the relationship between exercise and learning. 

Inspiring Exemplary Teaching and Learning: Perspectives on Teaching Academically Talented College Students edited by Larry Clark 
and John Zubizarreta (2008, 216pp). This rich collection of essays offers valuable insights into innovative teaching and significant learning in the 
context of academically challenging classrooms and programs. The volume provides theoretical, descriptive, and practical resources, including 
models of effective instructional practices, examples of successful courses designed for enhanced learning, and a list of online links to teaching 
and leaming centers and educational databases worldwide. 
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Occupy Honors Education edited by Lisa. Coleman, Jonathan D. Kotinek, and Alan Y. Oda (2017, 394pp). This collection of essays issues a 
call to honors to make diversity, equity, and inclusive excellence its central mission and ongoing state of mind. Echoing the AAC&U declaration 
‘without inclusion there is no true excellence,” the authors discuss transformational diversity, why itis essential, and how to achieve it. 

The Other Culture: Science and Mathematics Education in Honors edited by Ellen B. Buckner and Keith Garbutt (2012, 296pp). A collection 
of essays about teaching science and math in an honors context: topics include science in society, strategies for science and non-science 
majors, the threat of pseudoscience, chemistry, interdisciplinary science, scientific literacy, philosophy of science, thesis development, calculus, 
and statistics. 
Partners in the Parks: Field Guide to an Experiential Program in the National Parks by Joan Digby with reflective essays on theory and 
practice by student and faculty participants and National Park Service personnel (First Edition, 2010, 272pp). This monograph explores an 
experiential-learning program that fosters immersion in and stewardship of the national parks. The topics include program designs, group 
dynamics, philosophical and political issues, photography, wilderness exploration, and assessment. 

Partners in the Parks: Field Guide to an Experiential Program in the National Parks edited by Heather Thiessen-Reily and Joan Digby 
(Second Edition, 2016, 2689p), This collection of recent photographs and essays by students, faculty, and National Park Service rangers 
reflects upon PITP experiential-learning projects in new NPS locations, offers significant refinements in programming and curriculum for revisited 
projects, and provides strategies and tools for assessing PITP adventures. 

Place as Text: Approaches to Active Learning edited by Bernice Braid and Ada Long (Second Ecition, 2010, 128pp). Updated theory, 
information, and advice on experiential pedagogies developed within NCHC during the past 35 years, including Honors Semesters and City as 
Text™, along with suggested adaptations to multiple educational contexts. 

Preparing Tomorrow's Global Leaders: Honors International Education edited by Mary Kay Mulvaney and Kim Klein (2013, 4009p). A 
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development, assessment, funding, and security. The monograph also provides models of successful programs that incorporate high-impact 
educational practices, including City as Text™ pedagogy, service learning, and undergraduate research. 

Setting the Table for Diversity edited by Lisa L. Coleman and Jonathan D. Kotinek (2010, 288pp). This collection of essays provides definitions 
of diversity in honors, explores the challenges and opportunities diversity brings to honors education, and depicts the transformative nature of 
diversity when coupled with equity and inclusion. These essays discuss African American, Latinalo, international, and first-generation students 
as well as students with disabilities. Other issues include experiential and service learning, the politics of diversity, and the psychological 
resistance to it. Appendices relating to NCHC member institutions contain diversity statements and a structural diversity survey. 
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About The Demonstrable Value of 
Honors Education— 


“We all know— instinctively, experientially—that what we 
as honors teachers and administrators do for our students 
adds value to their college education and general college 
experience. Providing hard, demonstrable evidence for 
that which we know in our bodies as it were . . . turns 
out not to be so easy, a fact anyone who has had to 
make the case for additional, or even simply continued, 
honors funding to a new dean or college president has 
likely encountered. ‘The results presented in this volume 
provide, in a diversity of ways via a diversity of research 
approaches, the sorts of evidence honors teachers and 
administrators have long needed. Will that evidence be 
enough to convince every dean or college president of the 
need for continued honors sustenance? The answer may 
have to depend on the particular dean or president in 
question. I believe the essays in this monograph provide 
the strongest case for the added value of honors that has 
been made to date.” 
—Dr. Rusty Rushton 
University of Alabama at Birmingham 
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